{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "601cbd8a",
   "metadata": {
    "origin_pos": 0
   },
   "source": [
    "# 优化和深度学习\n",
    "\n",
    "本节将讨论优化与深度学习之间的关系以及在深度学习中使用优化的挑战。对于深度学习问题，我们通常会先定义*损失函数*。一旦我们有了损失函数，我们就可以使用优化算法来尝试最小化损失。在优化中，损失函数通常被称为优化问题的*目标函数*。按照传统惯例，大多数优化算法都关注的是*最小化*。如果我们需要最大化目标，那么有一个简单的解决方案：在目标函数前加负号即可。\n",
    "\n",
    "## 优化的目标\n",
    "\n",
    "尽管优化提供了一种最大限度地减少深度学习损失函数的方法，但本质上，优化和深度学习的目标是根本不同的。前者主要关注的是最小化目标，后者则关注在给定有限数据量的情况下寻找合适的模型。在 :numref:`sec_model_selection`中，我们详细讨论了这两个目标之间的区别。例如，训练误差和泛化误差通常不同：由于优化算法的目标函数通常是基于训练数据集的损失函数，因此优化的目标是减少训练误差。但是，深度学习（或更广义地说，统计推断）的目标是减少泛化误差。为了实现后者，除了使用优化算法来减少训练误差之外，我们还需要注意过拟合。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b78ba269",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:17.052244Z",
     "iopub.status.busy": "2022-12-07T16:35:17.051424Z",
     "iopub.status.idle": "2022-12-07T16:35:19.183942Z",
     "shell.execute_reply": "2022-12-07T16:35:19.183097Z"
    },
    "origin_pos": 2,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import torch\n",
    "from mpl_toolkits import mplot3d\n",
    "from d2l import torch as d2l"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b68c5c67",
   "metadata": {
    "origin_pos": 5
   },
   "source": [
    "为了说明上述不同的目标，引入两个概念*风险*和*经验风险*。如 :numref:`subsec_empirical-risk-and-risk`所述，经验风险是训练数据集的平均损失，而风险则是整个数据群的预期损失。下面我们定义了两个函数：风险函数`f`和经验风险函数`g`。假设我们只有有限的训练数据。因此，这里的`g`不如`f`平滑。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d81283f5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:19.187882Z",
     "iopub.status.busy": "2022-12-07T16:35:19.187326Z",
     "iopub.status.idle": "2022-12-07T16:35:19.192024Z",
     "shell.execute_reply": "2022-12-07T16:35:19.191038Z"
    },
    "origin_pos": 6,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return x * torch.cos(np.pi * x)\n",
    "\n",
    "def g(x):\n",
    "    return f(x) + 0.2 * torch.cos(5 * np.pi * x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4505b5e5",
   "metadata": {
    "origin_pos": 7
   },
   "source": [
    "下图说明，训练数据集的最低经验风险可能与最低风险（泛化误差）不同。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "437b12cc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:19.195611Z",
     "iopub.status.busy": "2022-12-07T16:35:19.194925Z",
     "iopub.status.idle": "2022-12-07T16:35:19.484910Z",
     "shell.execute_reply": "2022-12-07T16:35:19.484104Z"
    },
    "origin_pos": 8,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"261.023438pt\" height=\"183.35625pt\" viewBox=\"0 0 261.023438 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2022-12-07T16:35:19.431222</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 261.023438 183.35625 \n",
       "L 261.023438 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 58.523438 145.8 \n",
       "L 253.823438 145.8 \n",
       "L 253.823438 7.2 \n",
       "L 58.523438 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 85.334594 145.8 \n",
       "L 85.334594 7.2 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m2824ad9c34\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2824ad9c34\" x=\"85.334594\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0.6 -->\n",
       "      <g transform=\"translate(77.383032 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-36\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 121.202363 145.8 \n",
       "L 121.202363 7.2 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2824ad9c34\" x=\"121.202363\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 0.8 -->\n",
       "      <g transform=\"translate(113.2508 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-38\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 157.070131 145.8 \n",
       "L 157.070131 7.2 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2824ad9c34\" x=\"157.070131\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(149.118568 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 192.937899 145.8 \n",
       "L 192.937899 7.2 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2824ad9c34\" x=\"192.937899\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 1.2 -->\n",
       "      <g transform=\"translate(184.986337 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 228.805667 145.8 \n",
       "L 228.805667 7.2 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2824ad9c34\" x=\"228.805667\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 1.4 -->\n",
       "      <g transform=\"translate(220.854105 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-34\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- x -->\n",
       "     <g transform=\"translate(153.214063 174.076563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 58.523438 143.859356 \n",
       "L 253.823438 143.859356 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m51525d21a8\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m51525d21a8\" x=\"58.523438\" y=\"143.859356\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- −1.25 -->\n",
       "      <g transform=\"translate(20.878125 147.658575)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"179.199219\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"242.822266\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 58.523438 117.787485 \n",
       "L 253.823438 117.787485 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m51525d21a8\" x=\"58.523438\" y=\"117.787485\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- −1.00 -->\n",
       "      <g transform=\"translate(20.878125 121.586704)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"242.822266\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 58.523438 91.715614 \n",
       "L 253.823438 91.715614 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m51525d21a8\" x=\"58.523438\" y=\"91.715614\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- −0.75 -->\n",
       "      <g transform=\"translate(20.878125 95.514833)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-37\" d=\"M 525 4666 \n",
       "L 3525 4666 \n",
       "L 3525 4397 \n",
       "L 1831 0 \n",
       "L 1172 0 \n",
       "L 2766 4134 \n",
       "L 525 4134 \n",
       "L 525 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-37\" x=\"179.199219\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"242.822266\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 58.523438 65.643743 \n",
       "L 253.823438 65.643743 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m51525d21a8\" x=\"58.523438\" y=\"65.643743\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- −0.50 -->\n",
       "      <g transform=\"translate(20.878125 69.442961)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"242.822266\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 58.523438 39.571872 \n",
       "L 253.823438 39.571872 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m51525d21a8\" x=\"58.523438\" y=\"39.571872\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- −0.25 -->\n",
       "      <g transform=\"translate(20.878125 43.37109)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"179.199219\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"242.822266\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_21\">\n",
       "      <path d=\"M 58.523438 13.500001 \n",
       "L 253.823438 13.500001 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m51525d21a8\" x=\"58.523438\" y=\"13.500001\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.00 -->\n",
       "      <g transform=\"translate(29.257812 17.299219)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"159.033203\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- risk -->\n",
       "     <g transform=\"translate(14.798437 85.444531)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6b\" d=\"M 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 1991 \n",
       "L 2875 3500 \n",
       "L 3609 3500 \n",
       "L 1753 1863 \n",
       "L 3688 0 \n",
       "L 2938 0 \n",
       "L 1159 1709 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-72\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"41.113281\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"68.896484\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6b\" x=\"120.996094\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 67.40071 13.500003 \n",
       "L 69.194097 15.170636 \n",
       "L 70.987484 16.905096 \n",
       "L 72.78087 18.701587 \n",
       "L 74.574268 20.558178 \n",
       "L 76.367654 22.472793 \n",
       "L 78.161041 24.443255 \n",
       "L 79.954428 26.467279 \n",
       "L 81.747815 28.542444 \n",
       "L 83.541201 30.666212 \n",
       "L 85.334588 32.835959 \n",
       "L 87.127975 35.048933 \n",
       "L 88.921372 37.302291 \n",
       "L 90.714759 39.593065 \n",
       "L 92.508145 41.918217 \n",
       "L 94.301532 44.274585 \n",
       "L 96.09493 46.658991 \n",
       "L 97.888316 49.068064 \n",
       "L 99.681703 51.498428 \n",
       "L 101.47509 53.946613 \n",
       "L 103.268487 56.409067 \n",
       "L 105.061874 58.882175 \n",
       "L 106.85526 61.362264 \n",
       "L 108.648647 63.845598 \n",
       "L 110.442034 66.328375 \n",
       "L 112.235421 68.806789 \n",
       "L 114.028807 71.276954 \n",
       "L 115.822194 73.734941 \n",
       "L 117.615591 76.17683 \n",
       "L 119.408978 78.598597 \n",
       "L 121.202365 80.996283 \n",
       "L 122.995751 83.365866 \n",
       "L 124.789138 85.703332 \n",
       "L 126.582525 88.004638 \n",
       "L 128.375912 90.265759 \n",
       "L 130.169298 92.482695 \n",
       "L 131.962696 94.651446 \n",
       "L 133.756082 96.767987 \n",
       "L 135.549469 98.828367 \n",
       "L 137.342856 100.828657 \n",
       "L 139.136242 102.76496 \n",
       "L 140.929629 104.63342 \n",
       "L 142.723016 106.43021 \n",
       "L 144.516403 108.151587 \n",
       "L 146.3098 109.793847 \n",
       "L 148.103187 111.353359 \n",
       "L 149.896573 112.826543 \n",
       "L 151.68996 114.209911 \n",
       "L 153.483357 115.500065 \n",
       "L 155.276744 116.693665 \n",
       "L 157.070131 117.787485 \n",
       "L 158.863518 118.77838 \n",
       "L 160.656904 119.66333 \n",
       "L 162.450291 120.439388 \n",
       "L 164.243678 121.103756 \n",
       "L 166.037086 121.653712 \n",
       "L 167.830451 122.086682 \n",
       "L 169.623838 122.400218 \n",
       "L 171.417224 122.591982 \n",
       "L 173.210611 122.659811 \n",
       "L 175.003998 122.601617 \n",
       "L 176.797385 122.415522 \n",
       "L 178.590771 122.099736 \n",
       "L 180.384179 121.652667 \n",
       "L 182.177566 121.072837 \n",
       "L 183.970953 120.358952 \n",
       "L 185.764339 119.509869 \n",
       "L 187.557726 118.524593 \n",
       "L 189.351113 117.402347 \n",
       "L 191.1445 116.142447 \n",
       "L 192.937886 114.744421 \n",
       "L 194.731294 113.207964 \n",
       "L 196.524681 111.532958 \n",
       "L 198.318068 109.719441 \n",
       "L 200.111454 107.767642 \n",
       "L 201.904841 105.677966 \n",
       "L 203.698228 103.450997 \n",
       "L 205.491615 101.087523 \n",
       "L 207.285001 98.588417 \n",
       "L 209.078409 95.954889 \n",
       "L 210.871775 93.188257 \n",
       "L 212.665161 90.289945 \n",
       "L 214.458548 87.261663 \n",
       "L 216.251935 84.105268 \n",
       "L 218.045321 80.8228 \n",
       "L 219.838708 77.416484 \n",
       "L 221.632095 73.888707 \n",
       "L 223.425503 70.242011 \n",
       "L 225.21889 66.479238 \n",
       "L 227.012276 62.603333 \n",
       "L 228.805663 58.617261 \n",
       "L 230.59905 54.524358 \n",
       "L 232.392436 50.328058 \n",
       "L 234.185823 46.031953 \n",
       "L 235.97921 41.639804 \n",
       "L 237.772618 37.155464 \n",
       "L 239.566005 32.58314 \n",
       "L 241.359391 27.926982 \n",
       "L 243.152778 23.191436 \n",
       "L 244.946165 18.380867 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_24\">\n",
       "    <path d=\"M 67.40071 13.5 \n",
       "L 69.194097 18.433456 \n",
       "L 70.987484 23.350407 \n",
       "L 72.78087 28.170685 \n",
       "L 74.574268 32.817916 \n",
       "L 76.367654 37.221266 \n",
       "L 78.161041 41.31732 \n",
       "L 79.954428 45.051444 \n",
       "L 81.747815 48.3791 \n",
       "L 83.541201 51.266916 \n",
       "L 85.334588 53.693457 \n",
       "L 87.127975 55.649644 \n",
       "L 88.921372 57.138949 \n",
       "L 90.714759 58.177229 \n",
       "L 92.508145 58.792292 \n",
       "L 94.301532 59.023071 \n",
       "L 96.09493 58.918713 \n",
       "L 97.888316 58.537162 \n",
       "L 99.681703 57.943759 \n",
       "L 101.47509 57.209451 \n",
       "L 103.268487 56.409055 \n",
       "L 105.061874 55.619328 \n",
       "L 106.85526 54.916946 \n",
       "L 108.648647 54.376494 \n",
       "L 110.442034 54.068646 \n",
       "L 112.235421 54.058309 \n",
       "L 114.028807 54.402881 \n",
       "L 115.822194 55.15078 \n",
       "L 117.615591 56.340169 \n",
       "L 119.408978 57.997891 \n",
       "L 121.202365 60.138787 \n",
       "L 122.995751 62.765157 \n",
       "L 124.789138 65.866668 \n",
       "L 126.582525 69.420471 \n",
       "L 128.375912 73.391686 \n",
       "L 130.169298 77.734197 \n",
       "L 131.962696 82.39171 \n",
       "L 133.756082 87.298878 \n",
       "L 135.549469 92.383046 \n",
       "L 137.342856 97.565826 \n",
       "L 139.136242 102.764941 \n",
       "L 140.929629 107.896233 \n",
       "L 142.723016 112.875512 \n",
       "L 144.516403 117.620678 \n",
       "L 146.3098 122.053563 \n",
       "L 148.103187 126.101832 \n",
       "L 149.896573 129.700609 \n",
       "L 151.68996 132.794072 \n",
       "L 153.483357 135.336722 \n",
       "L 155.276744 137.29437 \n",
       "L 157.070131 138.644987 \n",
       "L 158.863518 139.379086 \n",
       "L 160.656904 139.5 \n",
       "L 162.450291 139.023567 \n",
       "L 164.243678 137.977834 \n",
       "L 166.037086 136.402173 \n",
       "L 167.830451 134.346423 \n",
       "L 169.623838 131.86934 \n",
       "L 171.417224 129.037322 \n",
       "L 173.210611 125.922699 \n",
       "L 175.003998 122.601667 \n",
       "L 176.797385 119.152746 \n",
       "L 178.590771 115.654464 \n",
       "L 180.384179 112.18357 \n",
       "L 182.177566 108.813114 \n",
       "L 183.970953 105.610479 \n",
       "L 185.764339 102.635803 \n",
       "L 187.557726 99.940426 \n",
       "L 189.351113 97.565701 \n",
       "L 191.1445 95.541747 \n",
       "L 192.937886 93.886925 \n",
       "L 194.731294 92.607258 \n",
       "L 196.524681 91.696301 \n",
       "L 198.318068 91.135274 \n",
       "L 200.111454 90.893576 \n",
       "L 201.904841 90.929493 \n",
       "L 203.698228 91.191274 \n",
       "L 205.491615 91.618389 \n",
       "L 207.285001 92.143071 \n",
       "L 209.078409 92.692069 \n",
       "L 210.871775 93.188232 \n",
       "L 212.665161 93.552752 \n",
       "L 214.458548 93.706965 \n",
       "L 216.251935 93.574352 \n",
       "L 218.045321 93.08251 \n",
       "L 219.838708 92.16492 \n",
       "L 221.632095 90.762742 \n",
       "L 223.425503 88.826172 \n",
       "L 225.21889 86.315895 \n",
       "L 227.012276 83.204038 \n",
       "L 228.805663 79.474757 \n",
       "L 230.59905 75.125067 \n",
       "L 232.392436 70.164721 \n",
       "L 234.185823 64.616123 \n",
       "L 235.97921 58.513901 \n",
       "L 237.772618 51.903946 \n",
       "L 239.566005 44.842872 \n",
       "L 241.359391 37.39609 \n",
       "L 243.152778 29.636758 \n",
       "L 244.946165 21.643698 \n",
       "\" clip-path=\"url(#paadc0583ab)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 58.523438 145.8 \n",
       "L 58.523438 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 253.823438 145.8 \n",
       "L 253.823438 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 58.523438 145.8 \n",
       "L 253.823438 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 58.523438 7.2 \n",
       "L 253.823438 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 138.679605 132.472942 \n",
       "Q 146.92526 135.240264 154.11098 137.651862 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "    <path d=\"M 150.955183 134.483116 \n",
       "L 154.11098 137.651862 \n",
       "L 149.682506 138.275252 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "   </g>\n",
       "   <g id=\"text_14\">\n",
       "    <!-- min of -->\n",
       "    <g transform=\"translate(67.40071 117.018421)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-6d\" d=\"M 3328 2828 \n",
       "Q 3544 3216 3844 3400 \n",
       "Q 4144 3584 4550 3584 \n",
       "Q 5097 3584 5394 3201 \n",
       "Q 5691 2819 5691 2113 \n",
       "L 5691 0 \n",
       "L 5113 0 \n",
       "L 5113 2094 \n",
       "Q 5113 2597 4934 2840 \n",
       "Q 4756 3084 4391 3084 \n",
       "Q 3944 3084 3684 2787 \n",
       "Q 3425 2491 3425 1978 \n",
       "L 3425 0 \n",
       "L 2847 0 \n",
       "L 2847 2094 \n",
       "Q 2847 2600 2669 2842 \n",
       "Q 2491 3084 2119 3084 \n",
       "Q 1678 3084 1418 2786 \n",
       "Q 1159 2488 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1356 3278 1631 3431 \n",
       "Q 1906 3584 2284 3584 \n",
       "Q 2666 3584 2933 3390 \n",
       "Q 3200 3197 3328 2828 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"97.412109\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"125.195312\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"188.574219\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6f\" x=\"220.361328\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-66\" x=\"281.542969\"/>\n",
       "    </g>\n",
       "    <!-- empirical risk -->\n",
       "    <g transform=\"translate(67.40071 128.216233)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"61.523438\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-70\" x=\"158.935547\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"222.412109\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-72\" x=\"250.195312\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"291.308594\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-63\" x=\"319.091797\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-61\" x=\"374.072266\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6c\" x=\"435.351562\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"463.134766\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-72\" x=\"494.921875\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"536.035156\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-73\" x=\"563.818359\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6b\" x=\"615.917969\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_8\">\n",
       "    <path d=\"M 174.551475 71.720249 \n",
       "Q 174.76892 96.361025 174.976499 119.883811 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "    <path d=\"M 176.941124 115.866318 \n",
       "L 174.976499 119.883811 \n",
       "L 172.94128 115.901615 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "   </g>\n",
       "   <g id=\"text_15\">\n",
       "    <!-- min of risk -->\n",
       "    <g transform=\"translate(148.103189 65.643743)scale(0.1 -0.1)\">\n",
       "     <use xlink:href=\"#DejaVuSans-6d\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"97.412109\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"125.195312\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"188.574219\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6f\" x=\"220.361328\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-66\" x=\"281.542969\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"316.748047\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-72\" x=\"348.535156\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"389.648438\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-73\" x=\"417.431641\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6b\" x=\"469.53125\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"paadc0583ab\">\n",
       "   <rect x=\"58.523438\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def annotate(text, xy, xytext):  #@save\n",
    "    d2l.plt.gca().annotate(text, xy=xy, xytext=xytext,\n",
    "                           arrowprops=dict(arrowstyle='->'))\n",
    "\n",
    "x = torch.arange(0.5, 1.5, 0.01)\n",
    "d2l.set_figsize((4.5, 2.5))\n",
    "d2l.plot(x, [f(x), g(x)], 'x', 'risk')\n",
    "annotate('min of\\nempirical risk', (1.0, -1.2), (0.5, -1.1))\n",
    "annotate('min of risk', (1.1, -1.05), (0.95, -0.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad772b35",
   "metadata": {
    "origin_pos": 10
   },
   "source": [
    "## 深度学习中的优化挑战\n",
    "\n",
    "本章将关注优化算法在最小化目标函数方面的性能，而不是模型的泛化误差。在 :numref:`sec_linear_regression`中，我们区分了优化问题中的解析解和数值解。在深度学习中，大多数目标函数都很复杂，没有解析解。相反，我们必须使用数值优化算法。本章中的优化算法都属于此类别。\n",
    "\n",
    "深度学习优化存在许多挑战。其中最令人烦恼的是局部最小值、鞍点和梯度消失。\n",
    "\n",
    "### 局部最小值\n",
    "\n",
    "对于任何目标函数$f(x)$，如果在$x$处对应的$f(x)$值小于在$x$附近任意其他点的$f(x)$值，那么$f(x)$可能是局部最小值。如果$f(x)$在$x$处的值是整个域中目标函数的最小值，那么$f(x)$是全局最小值。\n",
    "\n",
    "例如，给定函数\n",
    "\n",
    "$$f(x) = x \\cdot \\text{cos}(\\pi x) \\text{ for } -1.0 \\leq x \\leq 2.0,$$\n",
    "\n",
    "我们可以近似该函数的局部最小值和全局最小值。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ffe9bb03",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:19.489101Z",
     "iopub.status.busy": "2022-12-07T16:35:19.488551Z",
     "iopub.status.idle": "2022-12-07T16:35:19.726553Z",
     "shell.execute_reply": "2022-12-07T16:35:19.725755Z"
    },
    "origin_pos": 11,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"183.35625pt\" viewBox=\"0 0 245.120313 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2022-12-07T16:35:19.681731</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 245.120313 183.35625 \n",
       "L 245.120313 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 42.620312 145.8 \n",
       "L 237.920313 145.8 \n",
       "L 237.920313 7.2 \n",
       "L 42.620312 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.497585 145.8 \n",
       "L 51.497585 7.2 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m407c332b96\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m407c332b96\" x=\"51.497585\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(44.126491 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 110.877336 145.8 \n",
       "L 110.877336 7.2 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m407c332b96\" x=\"110.877336\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(107.696086 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 170.257086 145.8 \n",
       "L 170.257086 7.2 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m407c332b96\" x=\"170.257086\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(167.075836 160.398438)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 229.636837 145.8 \n",
       "L 229.636837 7.2 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m407c332b96\" x=\"229.636837\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(226.455587 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_5\">\n",
       "     <!-- x -->\n",
       "     <g transform=\"translate(137.310937 174.076563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 42.620312 137.560854 \n",
       "L 237.920313 137.560854 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <defs>\n",
       "       <path id=\"mfd93a33a28\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfd93a33a28\" x=\"42.620312\" y=\"137.560854\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(20.878125 141.360073)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 42.620312 96.055298 \n",
       "L 237.920313 96.055298 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfd93a33a28\" x=\"42.620312\" y=\"96.055298\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(29.257812 99.854516)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 42.620312 54.549741 \n",
       "L 237.920313 54.549741 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfd93a33a28\" x=\"42.620312\" y=\"54.549741\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(29.257812 58.34896)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 42.620312 13.044184 \n",
       "L 237.920313 13.044184 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mfd93a33a28\" x=\"42.620312\" y=\"13.044184\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(29.257812 16.843403)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- f(x) -->\n",
       "     <g transform=\"translate(14.798437 85.121094)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
       "Q 1566 4138 1362 3434 \n",
       "Q 1159 2731 1159 2009 \n",
       "Q 1159 1288 1364 580 \n",
       "Q 1569 -128 1984 -844 \n",
       "L 1484 -844 \n",
       "Q 1016 -109 783 600 \n",
       "Q 550 1309 550 2009 \n",
       "Q 550 2706 781 3412 \n",
       "Q 1013 4119 1484 4856 \n",
       "L 1984 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
       "L 1013 4856 \n",
       "Q 1481 4119 1714 3412 \n",
       "Q 1947 2706 1947 2009 \n",
       "Q 1947 1309 1714 600 \n",
       "Q 1481 -109 1013 -844 \n",
       "L 513 -844 \n",
       "Q 928 -128 1133 580 \n",
       "Q 1338 1288 1338 2009 \n",
       "Q 1338 2731 1133 3434 \n",
       "Q 928 4138 513 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_17\">\n",
       "    <path d=\"M 51.497585 54.549741 \n",
       "L 53.278976 55.973583 \n",
       "L 55.06037 57.731143 \n",
       "L 56.841761 59.784948 \n",
       "L 59.216953 62.915327 \n",
       "L 62.185941 67.318969 \n",
       "L 65.748726 73.060548 \n",
       "L 73.468093 85.670474 \n",
       "L 76.437078 90.068527 \n",
       "L 78.812269 93.246202 \n",
       "L 81.187462 96.055299 \n",
       "L 83.562652 98.448229 \n",
       "L 85.344043 99.948584 \n",
       "L 87.125435 101.185666 \n",
       "L 88.906828 102.15432 \n",
       "L 90.68822 102.853751 \n",
       "L 92.469613 103.287469 \n",
       "L 94.251006 103.463156 \n",
       "L 96.032398 103.392513 \n",
       "L 97.813791 103.091025 \n",
       "L 99.595183 102.577701 \n",
       "L 101.970373 101.602556 \n",
       "L 104.345563 100.350994 \n",
       "L 107.314551 98.50152 \n",
       "L 113.252526 94.408167 \n",
       "L 116.815311 92.107885 \n",
       "L 119.190501 90.797549 \n",
       "L 121.56569 89.747324 \n",
       "L 123.347084 89.168175 \n",
       "L 125.128476 88.793798 \n",
       "L 126.909869 88.644307 \n",
       "L 128.691262 88.736392 \n",
       "L 130.472654 89.083043 \n",
       "L 132.254047 89.693305 \n",
       "L 134.035439 90.572093 \n",
       "L 135.816832 91.720053 \n",
       "L 137.598223 93.133492 \n",
       "L 139.379615 94.804343 \n",
       "L 141.754805 97.410497 \n",
       "L 144.129996 100.410623 \n",
       "L 146.505187 103.750855 \n",
       "L 149.474176 108.303335 \n",
       "L 154.224555 116.092429 \n",
       "L 159.568731 124.791627 \n",
       "L 162.537715 129.195261 \n",
       "L 164.912907 132.325645 \n",
       "L 166.694301 134.379452 \n",
       "L 168.475695 136.137012 \n",
       "L 170.257086 137.560854 \n",
       "L 172.038477 138.61629 \n",
       "L 173.226071 139.099581 \n",
       "L 174.413672 139.396684 \n",
       "L 175.601266 139.5 \n",
       "L 176.78886 139.40277 \n",
       "L 177.976453 139.099165 \n",
       "L 179.164054 138.584272 \n",
       "L 180.351641 137.854217 \n",
       "L 181.539235 136.906143 \n",
       "L 182.726829 135.738243 \n",
       "L 184.508227 133.57304 \n",
       "L 186.289618 130.914408 \n",
       "L 188.071009 127.770565 \n",
       "L 189.852406 124.155588 \n",
       "L 191.633797 120.089484 \n",
       "L 194.008985 114.011594 \n",
       "L 196.384173 107.254707 \n",
       "L 199.353165 97.997842 \n",
       "L 202.915953 85.991267 \n",
       "L 210.041516 60.771447 \n",
       "L 214.198102 46.617515 \n",
       "L 217.167094 37.350755 \n",
       "L 219.542275 30.677498 \n",
       "L 221.323665 26.202348 \n",
       "L 223.105063 22.247434 \n",
       "L 224.886454 18.868262 \n",
       "L 226.074055 16.960752 \n",
       "L 227.261649 15.345879 \n",
       "L 228.449243 14.036461 \n",
       "L 229.04304 13.5 \n",
       "L 229.04304 13.5 \n",
       "\" clip-path=\"url(#p14d407bae2)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 42.620312 145.8 \n",
       "L 42.620312 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 237.920313 145.8 \n",
       "L 237.920313 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 42.620312 145.8 \n",
       "L 237.920313 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 42.620312 7.2 \n",
       "L 237.920313 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 99.3292 126.056399 \n",
       "Q 96.50034 117.196292 94.011534 109.401249 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "    <path d=\"M 93.322904 113.820049 \n",
       "L 94.011534 109.401249 \n",
       "L 97.133396 112.603433 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "   </g>\n",
       "   <g id=\"text_11\">\n",
       "    <!-- local minimum -->\n",
       "    <g transform=\"translate(65.154928 137.560854)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6d\" d=\"M 3328 2828 \n",
       "Q 3544 3216 3844 3400 \n",
       "Q 4144 3584 4550 3584 \n",
       "Q 5097 3584 5394 3201 \n",
       "Q 5691 2819 5691 2113 \n",
       "L 5691 0 \n",
       "L 5113 0 \n",
       "L 5113 2094 \n",
       "Q 5113 2597 4934 2840 \n",
       "Q 4756 3084 4391 3084 \n",
       "Q 3944 3084 3684 2787 \n",
       "Q 3425 2491 3425 1978 \n",
       "L 3425 0 \n",
       "L 2847 0 \n",
       "L 2847 2094 \n",
       "Q 2847 2600 2669 2842 \n",
       "Q 2491 3084 2119 3084 \n",
       "Q 1678 3084 1418 2786 \n",
       "Q 1159 2488 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1356 3278 1631 3431 \n",
       "Q 1906 3584 2284 3584 \n",
       "Q 2666 3584 2933 3390 \n",
       "Q 3200 3197 3328 2828 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-75\" d=\"M 544 1381 \n",
       "L 544 3500 \n",
       "L 1119 3500 \n",
       "L 1119 1403 \n",
       "Q 1119 906 1312 657 \n",
       "Q 1506 409 1894 409 \n",
       "Q 2359 409 2629 706 \n",
       "Q 2900 1003 2900 1516 \n",
       "L 2900 3500 \n",
       "L 3475 3500 \n",
       "L 3475 0 \n",
       "L 2900 0 \n",
       "L 2900 538 \n",
       "Q 2691 219 2414 64 \n",
       "Q 2138 -91 1772 -91 \n",
       "Q 1169 -91 856 284 \n",
       "Q 544 659 544 1381 \n",
       "z\n",
       "M 1991 3584 \n",
       "L 1991 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-6c\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6f\" x=\"27.783203\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-63\" x=\"88.964844\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-61\" x=\"143.945312\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6c\" x=\"205.224609\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"233.007812\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"264.794922\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"362.207031\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"389.990234\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"453.369141\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"481.152344\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-75\" x=\"578.564453\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"641.943359\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_8\">\n",
       "    <path d=\"M 185.798253 68.913244 \n",
       "Q 181.139381 101.210008 176.640134 132.400192 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "    <path d=\"M 179.190741 128.72672 \n",
       "L 176.640134 132.400192 \n",
       "L 175.23172 128.155623 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "   </g>\n",
       "   <g id=\"text_12\">\n",
       "    <!-- global minimum -->\n",
       "    <g transform=\"translate(146.505186 62.850852)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
       "Q 2906 2416 2648 2759 \n",
       "Q 2391 3103 1925 3103 \n",
       "Q 1463 3103 1205 2759 \n",
       "Q 947 2416 947 1791 \n",
       "Q 947 1169 1205 825 \n",
       "Q 1463 481 1925 481 \n",
       "Q 2391 481 2648 825 \n",
       "Q 2906 1169 2906 1791 \n",
       "z\n",
       "M 3481 434 \n",
       "Q 3481 -459 3084 -895 \n",
       "Q 2688 -1331 1869 -1331 \n",
       "Q 1566 -1331 1297 -1286 \n",
       "Q 1028 -1241 775 -1147 \n",
       "L 775 -588 \n",
       "Q 1028 -725 1275 -790 \n",
       "Q 1522 -856 1778 -856 \n",
       "Q 2344 -856 2625 -561 \n",
       "Q 2906 -266 2906 331 \n",
       "L 2906 616 \n",
       "Q 2728 306 2450 153 \n",
       "Q 2172 0 1784 0 \n",
       "Q 1141 0 747 490 \n",
       "Q 353 981 353 1791 \n",
       "Q 353 2603 747 3093 \n",
       "Q 1141 3584 1784 3584 \n",
       "Q 2172 3584 2450 3431 \n",
       "Q 2728 3278 2906 2969 \n",
       "L 2906 3500 \n",
       "L 3481 3500 \n",
       "L 3481 434 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-62\" d=\"M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "M 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2969 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-67\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6c\" x=\"63.476562\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6f\" x=\"91.259766\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-62\" x=\"152.441406\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-61\" x=\"215.917969\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6c\" x=\"277.197266\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"304.980469\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"336.767578\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"434.179688\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"461.962891\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"525.341797\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"553.125\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-75\" x=\"650.537109\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6d\" x=\"713.916016\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p14d407bae2\">\n",
       "   <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.arange(-1.0, 2.0, 0.01)\n",
    "d2l.plot(x, [f(x), ], 'x', 'f(x)')\n",
    "annotate('local minimum', (-0.3, -0.25), (-0.77, -1.0))\n",
    "annotate('global minimum', (1.1, -0.95), (0.6, 0.8))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ebb36fa",
   "metadata": {
    "origin_pos": 13
   },
   "source": [
    "深度学习模型的目标函数通常有许多局部最优解。当优化问题的数值解接近局部最优值时，随着目标函数解的梯度接近或变为零，通过最终迭代获得的数值解可能仅使目标函数*局部*最优，而不是*全局*最优。只有一定程度的噪声可能会使参数跳出局部最小值。事实上，这是小批量随机梯度下降的有利特性之一。在这种情况下，小批量上梯度的自然变化能够将参数从局部极小值中跳出。\n",
    "\n",
    "### 鞍点\n",
    "\n",
    "除了局部最小值之外，鞍点是梯度消失的另一个原因。*鞍点*（saddle point）是指函数的所有梯度都消失但既不是全局最小值也不是局部最小值的任何位置。考虑这个函数$f(x) = x^3$。它的一阶和二阶导数在$x=0$时消失。这时优化可能会停止，尽管它不是最小值。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "39c96094",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:19.730125Z",
     "iopub.status.busy": "2022-12-07T16:35:19.729556Z",
     "iopub.status.idle": "2022-12-07T16:35:19.924058Z",
     "shell.execute_reply": "2022-12-07T16:35:19.923282Z"
    },
    "origin_pos": 14,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"245.120313pt\" height=\"183.35625pt\" viewBox=\"0 0 245.120313 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2022-12-07T16:35:19.887265</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 245.120313 183.35625 \n",
       "L 245.120313 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 42.620312 145.8 \n",
       "L 237.920313 145.8 \n",
       "L 237.920313 7.2 \n",
       "L 42.620312 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.497585 145.8 \n",
       "L 51.497585 7.2 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m4c0989797a\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4c0989797a\" x=\"51.497585\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −2 -->\n",
       "      <g transform=\"translate(44.126491 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.995193 145.8 \n",
       "L 95.995193 7.2 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4c0989797a\" x=\"95.995193\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(88.624099 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 140.4928 145.8 \n",
       "L 140.4928 7.2 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4c0989797a\" x=\"140.4928\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(137.31155 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 184.990408 145.8 \n",
       "L 184.990408 7.2 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4c0989797a\" x=\"184.990408\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(181.809158 160.398438)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 229.488015 145.8 \n",
       "L 229.488015 7.2 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4c0989797a\" x=\"229.488015\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(226.306765 160.398438)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- x -->\n",
       "     <g transform=\"translate(137.310937 174.076563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 42.620312 115.697371 \n",
       "L 237.920313 115.697371 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"mdfae94f75d\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mdfae94f75d\" x=\"42.620312\" y=\"115.697371\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- −5 -->\n",
       "      <g transform=\"translate(20.878125 119.49659)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 42.620312 76.026324 \n",
       "L 237.920313 76.026324 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mdfae94f75d\" x=\"42.620312\" y=\"76.026324\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(29.257812 79.825543)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 42.620312 36.355276 \n",
       "L 237.920313 36.355276 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mdfae94f75d\" x=\"42.620312\" y=\"36.355276\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 5 -->\n",
       "      <g transform=\"translate(29.257812 40.154495)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- f(x) -->\n",
       "     <g transform=\"translate(14.798437 85.121094)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
       "Q 1566 4138 1362 3434 \n",
       "Q 1159 2731 1159 2009 \n",
       "Q 1159 1288 1364 580 \n",
       "Q 1569 -128 1984 -844 \n",
       "L 1484 -844 \n",
       "Q 1016 -109 783 600 \n",
       "Q 550 1309 550 2009 \n",
       "Q 550 2706 781 3412 \n",
       "Q 1013 4119 1484 4856 \n",
       "L 1984 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
       "L 1013 4856 \n",
       "Q 1481 4119 1714 3412 \n",
       "Q 1947 2706 1947 2009 \n",
       "Q 1947 1309 1714 600 \n",
       "Q 1481 -109 1013 -844 \n",
       "L 513 -844 \n",
       "Q 928 -128 1133 580 \n",
       "Q 1338 1288 1338 2009 \n",
       "Q 1338 2731 1133 3434 \n",
       "Q 928 4138 513 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_17\">\n",
       "    <path d=\"M 51.497585 139.5 \n",
       "L 54.61242 133.065804 \n",
       "L 57.72725 127.081819 \n",
       "L 60.842079 121.531714 \n",
       "L 63.956914 116.399137 \n",
       "L 67.071749 111.667781 \n",
       "L 70.186578 107.321324 \n",
       "L 73.301413 103.343416 \n",
       "L 76.416243 99.717747 \n",
       "L 79.531078 96.427977 \n",
       "L 82.645907 93.457784 \n",
       "L 85.760742 90.790833 \n",
       "L 88.875572 88.410803 \n",
       "L 91.990407 86.301357 \n",
       "L 95.105241 84.44617 \n",
       "L 98.220074 82.828916 \n",
       "L 101.779882 81.251025 \n",
       "L 105.339689 79.938199 \n",
       "L 108.899497 78.866065 \n",
       "L 112.459308 78.010249 \n",
       "L 116.464091 77.275676 \n",
       "L 120.913853 76.702192 \n",
       "L 125.808589 76.311456 \n",
       "L 131.593279 76.089797 \n",
       "L 140.047824 76.026332 \n",
       "L 151.172226 75.916641 \n",
       "L 156.956915 75.624432 \n",
       "L 161.851651 75.148864 \n",
       "L 166.301412 74.478265 \n",
       "L 170.306195 73.640008 \n",
       "L 173.866006 72.679079 \n",
       "L 177.425814 71.489646 \n",
       "L 180.985622 70.047334 \n",
       "L 184.100454 68.55871 \n",
       "L 187.215286 66.841486 \n",
       "L 190.330121 64.879331 \n",
       "L 193.444956 62.655918 \n",
       "L 196.559785 60.154922 \n",
       "L 199.674615 57.360012 \n",
       "L 202.78945 54.254853 \n",
       "L 205.904285 50.823123 \n",
       "L 209.019114 47.048497 \n",
       "L 212.133949 42.914636 \n",
       "L 215.248779 38.405226 \n",
       "L 218.363614 33.50392 \n",
       "L 221.478443 28.194408 \n",
       "L 224.593278 22.460342 \n",
       "L 227.708108 16.285413 \n",
       "L 229.04304 13.5 \n",
       "L 229.04304 13.5 \n",
       "\" clip-path=\"url(#p88ec753ee7)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 42.620312 145.8 \n",
       "L 42.620312 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 237.920313 145.8 \n",
       "L 237.920313 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 42.620313 145.8 \n",
       "L 237.920313 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 42.620313 7.2 \n",
       "L 237.920313 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 146.262941 104.137265 \n",
       "Q 143.590236 91.851413 141.155193 80.658044 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "    <path d=\"M 140.051188 84.991769 \n",
       "L 141.155193 80.658044 \n",
       "L 143.95977 84.141483 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "   </g>\n",
       "   <g id=\"text_11\">\n",
       "    <!-- saddle point -->\n",
       "    <g transform=\"translate(117.354044 115.697371)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \n",
       "L 2906 4863 \n",
       "L 3481 4863 \n",
       "L 3481 0 \n",
       "L 2906 0 \n",
       "L 2906 525 \n",
       "Q 2725 213 2448 61 \n",
       "Q 2172 -91 1784 -91 \n",
       "Q 1150 -91 751 415 \n",
       "Q 353 922 353 1747 \n",
       "Q 353 2572 751 3078 \n",
       "Q 1150 3584 1784 3584 \n",
       "Q 2172 3584 2448 3432 \n",
       "Q 2725 3281 2906 2969 \n",
       "z\n",
       "M 947 1747 \n",
       "Q 947 1113 1208 752 \n",
       "Q 1469 391 1925 391 \n",
       "Q 2381 391 2643 752 \n",
       "Q 2906 1113 2906 1747 \n",
       "Q 2906 2381 2643 2742 \n",
       "Q 2381 3103 1925 3103 \n",
       "Q 1469 3103 1208 2742 \n",
       "Q 947 2381 947 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-73\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-61\" x=\"52.099609\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-64\" x=\"113.378906\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-64\" x=\"176.855469\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6c\" x=\"240.332031\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-65\" x=\"268.115234\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"329.638672\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-70\" x=\"361.425781\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6f\" x=\"424.902344\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"486.083984\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"513.867188\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-74\" x=\"577.246094\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p88ec753ee7\">\n",
       "   <rect x=\"42.620312\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.arange(-2.0, 2.0, 0.01)\n",
    "d2l.plot(x, [x**3], 'x', 'f(x)')\n",
    "annotate('saddle point', (0, -0.2), (-0.52, -5.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c74476ac",
   "metadata": {
    "origin_pos": 16
   },
   "source": [
    "如下例所示，较高维度的鞍点甚至更加隐蔽。考虑这个函数$f(x, y) = x^2 - y^2$。它的鞍点为$(0, 0)$。这是关于$y$的最大值，也是关于$x$的最小值。此外，它看起来像个马鞍，这就是鞍点的名字由来。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ea700d14",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:19.929274Z",
     "iopub.status.busy": "2022-12-07T16:35:19.928717Z",
     "iopub.status.idle": "2022-12-07T16:35:20.063326Z",
     "shell.execute_reply": "2022-12-07T16:35:20.062448Z"
    },
    "origin_pos": 18,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/d2l-worker/miniconda3/envs/d2l-zh-release-1/lib/python3.9/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2895.)\n",
      "  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"169.715408pt\" height=\"171.586921pt\" viewBox=\"0 0 169.715408 171.586921\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2022-12-07T16:35:20.024712</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 171.586921 \n",
       "L 169.715408 171.586921 \n",
       "L 169.715408 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"patch_2\">\n",
       "   <path d=\"M 7.2 145.8 \n",
       "L 145.8 145.8 \n",
       "L 145.8 7.2 \n",
       "L 7.2 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"pane3d_1\">\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 17.665256 111.625682 \n",
       "L 63.435502 73.260154 \n",
       "L 62.799252 17.930204 \n",
       "L 14.838663 52.929657 \n",
       "\" style=\"fill: #f2f2f2; opacity: 0.5; stroke: #f2f2f2; stroke-linejoin: miter\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"pane3d_2\">\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 63.435502 73.260154 \n",
       "L 136.880417 94.607691 \n",
       "L 139.501407 37.371964 \n",
       "L 62.799252 17.930204 \n",
       "\" style=\"fill: #e6e6e6; opacity: 0.5; stroke: #e6e6e6; stroke-linejoin: miter\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"pane3d_3\">\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 17.665256 111.625682 \n",
       "L 95.520453 137.053189 \n",
       "L 136.880417 94.607691 \n",
       "L 63.435502 73.260154 \n",
       "\" style=\"fill: #ececec; opacity: 0.5; stroke: #ececec; stroke-linejoin: miter\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axis3d_1\">\n",
       "   <g id=\"line2d_1\">\n",
       "    <path d=\"M 17.665256 111.625682 \n",
       "L 95.520453 137.053189 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"text_1\">\n",
       "    <!-- x -->\n",
       "    <g transform=\"translate(36.190346 162.307234)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_1\">\n",
       "    <path d=\"M 22.380575 113.165705 \n",
       "L 67.902197 74.558445 \n",
       "L 67.454802 19.11025 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 55.566605 124.004262 \n",
       "L 99.270858 83.676077 \n",
       "L 100.18343 27.406003 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 90.315667 135.353306 \n",
       "L 131.990658 93.186431 \n",
       "L 134.384718 36.075033 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "   </g>\n",
       "   <g id=\"xtick_1\">\n",
       "    <g id=\"line2d_2\">\n",
       "     <path d=\"M 22.776975 112.829515 \n",
       "L 21.586075 113.839527 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_2\">\n",
       "     <!-- −1 -->\n",
       "     <g transform=\"translate(7.454595 136.528767)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_2\">\n",
       "    <g id=\"line2d_3\">\n",
       "     <path d=\"M 55.94789 123.65243 \n",
       "L 54.80236 124.709471 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_3\">\n",
       "     <!-- 0 -->\n",
       "     <g transform=\"translate(44.898548 147.812111)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_3\">\n",
       "    <g id=\"line2d_4\">\n",
       "     <path d=\"M 90.679959 134.984714 \n",
       "L 89.585446 136.092146 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_4\">\n",
       "     <!-- 1 -->\n",
       "     <g transform=\"translate(79.730646 159.630877)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axis3d_2\">\n",
       "   <g id=\"line2d_5\">\n",
       "    <path d=\"M 136.880417 94.607691 \n",
       "L 95.520453 137.053189 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"text_5\">\n",
       "    <!-- y -->\n",
       "    <g transform=\"translate(141.167797 145.683654)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-79\" d=\"M 2059 -325 \n",
       "Q 1816 -950 1584 -1140 \n",
       "Q 1353 -1331 966 -1331 \n",
       "L 506 -1331 \n",
       "L 506 -850 \n",
       "L 844 -850 \n",
       "Q 1081 -850 1212 -737 \n",
       "Q 1344 -625 1503 -206 \n",
       "L 1606 56 \n",
       "L 191 3500 \n",
       "L 800 3500 \n",
       "L 1894 763 \n",
       "L 2988 3500 \n",
       "L 3597 3500 \n",
       "L 2059 -325 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-79\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_2\">\n",
       "    <path d=\"M 18.155257 50.509358 \n",
       "L 20.819213 108.981973 \n",
       "L 98.382242 134.116289 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 39.885822 34.651381 \n",
       "L 41.525154 91.625842 \n",
       "L 117.126847 114.879713 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 59.991995 19.978812 \n",
       "L 60.746858 75.513829 \n",
       "L 134.460919 97.090691 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "   </g>\n",
       "   <g id=\"xtick_4\">\n",
       "    <g id=\"line2d_6\">\n",
       "     <path d=\"M 97.728616 133.904481 \n",
       "L 99.691184 134.540452 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- −1 -->\n",
       "     <g transform=\"translate(103.898882 155.145417)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_5\">\n",
       "    <g id=\"line2d_7\">\n",
       "     <path d=\"M 116.491011 114.68414 \n",
       "L 118.400099 115.271345 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- 0 -->\n",
       "     <g transform=\"translate(126.349632 135.361913)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_6\">\n",
       "    <g id=\"line2d_8\">\n",
       "     <path d=\"M 133.842096 96.909555 \n",
       "L 135.700045 97.453396 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_8\">\n",
       "     <!-- 1 -->\n",
       "     <g transform=\"translate(143.234942 117.068702)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axis3d_3\">\n",
       "   <g id=\"line2d_9\">\n",
       "    <path d=\"M 136.880417 94.607691 \n",
       "L 139.501407 37.371964 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_3\">\n",
       "    <path d=\"M 136.930632 93.511118 \n",
       "L 63.423287 72.197896 \n",
       "L 17.611191 110.502989 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 138.162202 66.616789 \n",
       "L 63.124011 46.172077 \n",
       "L 16.284113 82.945357 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "    <path d=\"M 139.446688 38.566888 \n",
       "L 62.812508 19.082955 \n",
       "L 14.897772 54.157093 \n",
       "\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8\"/>\n",
       "   </g>\n",
       "   <g id=\"xtick_7\">\n",
       "    <g id=\"line2d_10\">\n",
       "     <path d=\"M 136.313678 93.332234 \n",
       "L 138.166009 93.869311 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_9\">\n",
       "     <!-- −1 -->\n",
       "     <g transform=\"translate(147.773221 98.515807)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_8\">\n",
       "    <g id=\"line2d_11\">\n",
       "     <path d=\"M 137.531779 66.445026 \n",
       "L 139.424579 66.960733 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- 0 -->\n",
       "     <g transform=\"translate(153.568228 71.725004)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"xtick_9\">\n",
       "    <g id=\"line2d_12\">\n",
       "     <path d=\"M 138.802197 38.403029 \n",
       "L 140.737271 38.895014 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- 1 -->\n",
       "     <g transform=\"translate(155.241948 43.789973)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"line2d_13\">\n",
       "    <defs>\n",
       "     <path id=\"md213eef3e3\" d=\"M -3 3 \n",
       "L 3 -3 \n",
       "M -3 -3 \n",
       "L 3 3 \n",
       "\" style=\"stroke: #ff0000\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#p5979f259ce)\">\n",
       "     <use xlink:href=\"#md213eef3e3\" x=\"78.372973\" y=\"74.627027\" style=\"fill: #ff0000; stroke: #ff0000\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"Line3DCollection_4\">\n",
       "    <path d=\"M 24.318635 81.868935 \n",
       "L 24.710556 80.407083 \n",
       "L 25.103462 78.967565 \n",
       "L 25.497331 77.550469 \n",
       "L 25.892142 76.155867 \n",
       "L 26.287877 74.783829 \n",
       "L 26.684512 73.434437 \n",
       "L 27.082029 72.107753 \n",
       "L 27.480405 70.803848 \n",
       "L 27.879621 69.522781 \n",
       "L 28.279656 68.264617 \n",
       "L 28.680486 67.029418 \n",
       "L 29.082092 65.817241 \n",
       "L 29.484454 64.628137 \n",
       "L 29.887548 63.462163 \n",
       "L 30.291353 62.319369 \n",
       "L 30.69585 61.199798 \n",
       "L 31.101015 60.103501 \n",
       "L 31.506827 59.030517 \n",
       "L 31.913264 57.980887 \n",
       "L 32.320305 56.954651 \n",
       "L 32.727928 55.951842 \n",
       "L 33.136113 54.972491 \n",
       "L 33.544835 54.016632 \n",
       "L 33.954075 53.084291 \n",
       "L 34.363809 52.175495 \n",
       "L 34.774016 51.290264 \n",
       "L 35.184675 50.42862 \n",
       "L 35.595762 49.590584 \n",
       "L 36.007258 48.776166 \n",
       "L 36.419139 47.985381 \n",
       "L 36.831383 47.218241 \n",
       "L 37.243968 46.474752 \n",
       "L 37.656874 45.75492 \n",
       "L 38.070078 45.058749 \n",
       "L 38.483558 44.386238 \n",
       "L 38.897292 43.737386 \n",
       "L 39.311258 43.112188 \n",
       "L 39.725435 42.510639 \n",
       "L 40.1398 41.932727 \n",
       "L 40.554333 41.378444 \n",
       "L 40.96901 40.847773 \n",
       "L 41.383811 40.340698 \n",
       "L 41.798714 39.857202 \n",
       "L 42.213697 39.39726 \n",
       "L 42.628739 38.960852 \n",
       "L 43.043818 38.547951 \n",
       "L 43.458913 38.158529 \n",
       "L 43.874002 37.792556 \n",
       "L 44.289064 37.449996 \n",
       "L 44.704077 37.130816 \n",
       "L 45.119022 36.834979 \n",
       "L 45.533875 36.562444 \n",
       "L 45.948617 36.313168 \n",
       "L 46.363227 36.087108 \n",
       "L 46.777683 35.884218 \n",
       "L 47.191965 35.704448 \n",
       "L 47.606052 35.547748 \n",
       "L 48.019923 35.414064 \n",
       "L 48.433558 35.30334 \n",
       "L 48.846937 35.215521 \n",
       "L 49.260038 35.150544 \n",
       "L 49.672843 35.108352 \n",
       "L 50.08533 35.088876 \n",
       "L 50.497481 35.092054 \n",
       "L 50.909274 35.117817 \n",
       "L 51.32069 35.166094 \n",
       "L 51.731711 35.236814 \n",
       "L 52.142314 35.329904 \n",
       "L 52.552483 35.445287 \n",
       "L 52.962197 35.582887 \n",
       "L 53.371437 35.742621 \n",
       "L 53.780184 35.924412 \n",
       "L 54.18842 36.128173 \n",
       "L 54.596125 36.35382 \n",
       "L 55.003281 36.601267 \n",
       "L 55.40987 36.870424 \n",
       "L 55.815872 37.161202 \n",
       "L 56.221272 37.473507 \n",
       "L 56.626048 37.807245 \n",
       "L 57.030186 38.162323 \n",
       "L 57.433665 38.538641 \n",
       "L 57.83647 38.936101 \n",
       "L 58.238581 39.354604 \n",
       "L 58.639982 39.794045 \n",
       "L 59.040658 40.254323 \n",
       "L 59.440588 40.73533 \n",
       "L 59.839758 41.236963 \n",
       "L 60.23815 41.759111 \n",
       "L 60.635749 42.301665 \n",
       "L 61.032539 42.864518 \n",
       "L 61.428501 43.447552 \n",
       "L 61.823622 44.050655 \n",
       "L 62.217886 44.673714 \n",
       "L 62.611275 45.316612 \n",
       "L 63.003776 45.979229 \n",
       "L 63.395374 46.661452 \n",
       "L 63.786051 47.363154 \n",
       "L 64.175795 48.084218 \n",
       "L 64.564591 48.824522 \n",
       "L 64.952423 49.583939 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 31.326259 93.962164 \n",
       "L 31.717396 92.505925 \n",
       "L 32.109392 91.071739 \n",
       "L 32.502226 89.659692 \n",
       "L 32.895877 88.269855 \n",
       "L 33.290328 86.902299 \n",
       "L 33.685555 85.557103 \n",
       "L 34.081538 84.234331 \n",
       "L 34.478257 82.934053 \n",
       "L 34.875691 81.656327 \n",
       "L 35.27382 80.401219 \n",
       "L 35.672621 79.168789 \n",
       "L 36.072075 77.959094 \n",
       "L 36.47216 76.772186 \n",
       "L 36.872855 75.608121 \n",
       "L 37.274138 74.46695 \n",
       "L 37.675991 73.348714 \n",
       "L 38.078388 72.253465 \n",
       "L 38.481312 71.18124 \n",
       "L 38.884739 70.132086 \n",
       "L 39.288648 69.106036 \n",
       "L 39.693019 68.103128 \n",
       "L 40.09783 67.123389 \n",
       "L 40.503058 66.166858 \n",
       "L 40.908684 65.233556 \n",
       "L 41.314685 64.323513 \n",
       "L 41.72104 63.436752 \n",
       "L 42.127727 62.57329 \n",
       "L 42.534725 61.733151 \n",
       "L 42.942013 60.916346 \n",
       "L 43.349569 60.122888 \n",
       "L 43.757371 59.352793 \n",
       "L 44.165398 58.606065 \n",
       "L 44.573629 57.882711 \n",
       "L 44.982042 57.182736 \n",
       "L 45.390616 56.506138 \n",
       "L 45.799329 55.852921 \n",
       "L 46.208161 55.223076 \n",
       "L 46.61709 54.6166 \n",
       "L 47.026094 54.033483 \n",
       "L 47.435153 53.473716 \n",
       "L 47.844244 52.937285 \n",
       "L 48.253348 52.424173 \n",
       "L 48.662443 51.934363 \n",
       "L 49.071508 51.467837 \n",
       "L 49.480521 51.024568 \n",
       "L 49.889463 50.604535 \n",
       "L 50.298312 50.207709 \n",
       "L 50.707048 49.83406 \n",
       "L 51.115649 49.483556 \n",
       "L 51.524095 49.156164 \n",
       "L 51.932366 48.851848 \n",
       "L 52.340442 48.570568 \n",
       "L 52.748301 48.312282 \n",
       "L 53.155923 48.076949 \n",
       "L 53.563289 47.864524 \n",
       "L 53.970379 47.674958 \n",
       "L 54.377171 47.508201 \n",
       "L 54.783646 47.364204 \n",
       "L 55.189785 47.242911 \n",
       "L 55.595568 47.144266 \n",
       "L 56.000974 47.068209 \n",
       "L 56.405986 47.014685 \n",
       "L 56.810583 46.983627 \n",
       "L 57.214746 46.974972 \n",
       "L 57.618456 46.988655 \n",
       "L 58.021694 47.024606 \n",
       "L 58.424441 47.082757 \n",
       "L 58.826678 47.163032 \n",
       "L 59.228387 47.265361 \n",
       "L 59.629549 47.389666 \n",
       "L 60.030146 47.535868 \n",
       "L 60.430159 47.70389 \n",
       "L 60.829571 47.893648 \n",
       "L 61.228364 48.10506 \n",
       "L 61.626519 48.33804 \n",
       "L 62.02402 48.592501 \n",
       "L 62.420848 48.868355 \n",
       "L 62.816987 49.165512 \n",
       "L 63.212419 49.483878 \n",
       "L 63.607128 49.823362 \n",
       "L 64.001094 50.183866 \n",
       "L 64.394305 50.565296 \n",
       "L 64.78674 50.96755 \n",
       "L 65.178384 51.39053 \n",
       "L 65.569223 51.834134 \n",
       "L 65.959238 52.298258 \n",
       "L 66.348414 52.782799 \n",
       "L 66.736736 53.287649 \n",
       "L 67.124187 53.812701 \n",
       "L 67.510754 54.357851 \n",
       "L 67.896419 54.922981 \n",
       "L 68.281169 55.507983 \n",
       "L 68.664989 56.112745 \n",
       "L 69.047862 56.737151 \n",
       "L 69.429775 57.381086 \n",
       "L 69.810715 58.044436 \n",
       "L 70.190665 58.727079 \n",
       "L 70.569613 59.428898 \n",
       "L 70.947545 60.149774 \n",
       "L 71.324447 60.889583 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 38.218828 103.780253 \n",
       "L 38.609854 102.326915 \n",
       "L 39.001619 100.895427 \n",
       "L 39.3941 99.485873 \n",
       "L 39.787279 98.098326 \n",
       "L 40.181136 96.732856 \n",
       "L 40.575648 95.389541 \n",
       "L 40.970796 94.068445 \n",
       "L 41.366559 92.769638 \n",
       "L 41.762917 91.493179 \n",
       "L 42.159849 90.239131 \n",
       "L 42.557334 89.007557 \n",
       "L 42.95535 87.798513 \n",
       "L 43.353879 86.61205 \n",
       "L 43.752897 85.448224 \n",
       "L 44.152384 84.307087 \n",
       "L 44.552321 83.18868 \n",
       "L 44.952684 82.093055 \n",
       "L 45.353454 81.020249 \n",
       "L 45.754608 79.970308 \n",
       "L 46.156126 78.943266 \n",
       "L 46.557987 77.939162 \n",
       "L 46.960171 76.958022 \n",
       "L 47.362654 75.999885 \n",
       "L 47.765417 75.064773 \n",
       "L 48.168439 74.152716 \n",
       "L 48.571696 73.263738 \n",
       "L 48.97517 72.397856 \n",
       "L 49.378839 71.555092 \n",
       "L 49.782681 70.73546 \n",
       "L 50.186676 69.938975 \n",
       "L 50.590802 69.165649 \n",
       "L 50.995038 68.415489 \n",
       "L 51.399364 67.688502 \n",
       "L 51.803758 66.984692 \n",
       "L 52.208199 66.304062 \n",
       "L 52.612667 65.646611 \n",
       "L 53.01714 65.012334 \n",
       "L 53.421598 64.401226 \n",
       "L 53.826019 63.813282 \n",
       "L 54.230384 63.248489 \n",
       "L 54.63467 62.706835 \n",
       "L 55.038858 62.188305 \n",
       "L 55.442928 61.692882 \n",
       "L 55.846858 61.220547 \n",
       "L 56.250628 60.771277 \n",
       "L 56.654218 60.345048 \n",
       "L 57.057607 59.941835 \n",
       "L 57.460775 59.561607 \n",
       "L 57.863703 59.204334 \n",
       "L 58.266369 58.869982 \n",
       "L 58.668754 58.558517 \n",
       "L 59.070839 58.269899 \n",
       "L 59.472602 58.00409 \n",
       "L 59.874026 57.761047 \n",
       "L 60.275089 57.540725 \n",
       "L 60.675774 57.343078 \n",
       "L 61.076059 57.168058 \n",
       "L 61.475927 57.015614 \n",
       "L 61.875356 56.885692 \n",
       "L 62.27433 56.778238 \n",
       "L 62.672828 56.693194 \n",
       "L 63.070832 56.630501 \n",
       "L 63.468323 56.590099 \n",
       "L 63.865283 56.571923 \n",
       "L 64.261693 56.57591 \n",
       "L 64.657535 56.601991 \n",
       "L 65.052791 56.650098 \n",
       "L 65.447442 56.720159 \n",
       "L 65.841471 56.812102 \n",
       "L 66.23486 56.925851 \n",
       "L 66.627591 57.06133 \n",
       "L 67.019647 57.218461 \n",
       "L 67.41101 57.397162 \n",
       "L 67.801664 57.597353 \n",
       "L 68.191591 57.818949 \n",
       "L 68.580774 58.061863 \n",
       "L 68.969197 58.32601 \n",
       "L 69.356843 58.6113 \n",
       "L 69.743695 58.917641 \n",
       "L 70.129739 59.244942 \n",
       "L 70.514955 59.593109 \n",
       "L 70.899331 59.962046 \n",
       "L 71.282848 60.351656 \n",
       "L 71.665492 60.761839 \n",
       "L 72.047248 61.192496 \n",
       "L 72.428099 61.643524 \n",
       "L 72.808031 62.114823 \n",
       "L 73.187029 62.606284 \n",
       "L 73.565078 63.117802 \n",
       "L 73.942165 63.649275 \n",
       "L 74.318273 64.200585 \n",
       "L 74.693388 64.771628 \n",
       "L 75.067499 65.362292 \n",
       "L 75.440588 65.972463 \n",
       "L 75.812644 66.602026 \n",
       "L 76.183653 67.25087 \n",
       "L 76.553601 67.918873 \n",
       "L 76.922474 68.605921 \n",
       "L 77.290262 69.311896 \n",
       "L 77.656949 70.036674 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 45.032744 111.384675 \n",
       "L 45.424332 109.931559 \n",
       "L 45.816542 108.500163 \n",
       "L 46.20935 107.090575 \n",
       "L 46.602737 105.702867 \n",
       "L 46.996684 104.337108 \n",
       "L 47.391169 102.993377 \n",
       "L 47.786171 101.671739 \n",
       "L 48.18167 100.372262 \n",
       "L 48.577645 99.095005 \n",
       "L 48.974078 97.840034 \n",
       "L 49.370944 96.607409 \n",
       "L 49.768224 95.397187 \n",
       "L 50.165899 94.20942 \n",
       "L 50.563946 93.044164 \n",
       "L 50.962344 91.90147 \n",
       "L 51.361073 90.78138 \n",
       "L 51.760112 89.683946 \n",
       "L 52.15944 88.609204 \n",
       "L 52.559036 87.557202 \n",
       "L 52.958878 86.527975 \n",
       "L 53.358946 85.521559 \n",
       "L 53.759221 84.537983 \n",
       "L 54.159678 83.577285 \n",
       "L 54.5603 82.639488 \n",
       "L 54.961064 81.724622 \n",
       "L 55.361948 80.83271 \n",
       "L 55.762933 79.963771 \n",
       "L 56.163997 79.117827 \n",
       "L 56.56512 78.294892 \n",
       "L 56.966281 77.49498 \n",
       "L 57.367458 76.718105 \n",
       "L 57.768632 75.964275 \n",
       "L 58.169781 75.233496 \n",
       "L 58.570885 74.525773 \n",
       "L 58.971924 73.841108 \n",
       "L 59.372875 73.179502 \n",
       "L 59.77372 72.54095 \n",
       "L 60.174438 71.925448 \n",
       "L 60.575007 71.332989 \n",
       "L 60.975408 70.763564 \n",
       "L 61.37562 70.217159 \n",
       "L 61.775624 69.69376 \n",
       "L 62.175399 69.193351 \n",
       "L 62.574924 68.715913 \n",
       "L 62.97418 68.261423 \n",
       "L 63.373147 67.829859 \n",
       "L 63.771806 67.421194 \n",
       "L 64.170136 67.0354 \n",
       "L 64.568117 66.672446 \n",
       "L 64.965731 66.3323 \n",
       "L 65.362957 66.014927 \n",
       "L 65.759776 65.72029 \n",
       "L 66.15617 65.448348 \n",
       "L 66.552118 65.19906 \n",
       "L 66.947602 64.972384 \n",
       "L 67.342604 64.768271 \n",
       "L 67.737102 64.586676 \n",
       "L 68.131081 64.427547 \n",
       "L 68.52452 64.290832 \n",
       "L 68.917401 64.176477 \n",
       "L 69.309705 64.084425 \n",
       "L 69.701416 64.014617 \n",
       "L 70.092514 63.966994 \n",
       "L 70.482982 63.941492 \n",
       "L 70.872802 63.938048 \n",
       "L 71.261955 63.956594 \n",
       "L 71.650425 63.997062 \n",
       "L 72.038193 64.059383 \n",
       "L 72.425245 64.143483 \n",
       "L 72.81156 64.249288 \n",
       "L 73.197123 64.376723 \n",
       "L 73.581917 64.525709 \n",
       "L 73.965926 64.696167 \n",
       "L 74.349132 64.888016 \n",
       "L 74.731519 65.101172 \n",
       "L 75.113072 65.335551 \n",
       "L 75.493773 65.591065 \n",
       "L 75.873609 65.867628 \n",
       "L 76.252561 66.165146 \n",
       "L 76.630616 66.483532 \n",
       "L 77.007756 66.82269 \n",
       "L 77.383969 67.182526 \n",
       "L 77.759237 67.562943 \n",
       "L 78.133546 67.963842 \n",
       "L 78.506883 68.385127 \n",
       "L 78.879231 68.826692 \n",
       "L 79.250577 69.28844 \n",
       "L 79.620906 69.770263 \n",
       "L 79.990205 70.272055 \n",
       "L 80.35846 70.793717 \n",
       "L 80.725657 71.335132 \n",
       "L 81.091781 71.896195 \n",
       "L 81.456822 72.476795 \n",
       "L 81.820764 73.076819 \n",
       "L 82.183595 73.696154 \n",
       "L 82.545304 74.334689 \n",
       "L 82.905875 74.992303 \n",
       "L 83.265298 75.668883 \n",
       "L 83.62356 76.36431 \n",
       "L 83.980649 77.078464 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 51.803488 116.813651 \n",
       "L 52.196318 115.358076 \n",
       "L 52.589653 113.92417 \n",
       "L 52.983469 112.512019 \n",
       "L 53.377749 111.121697 \n",
       "L 53.772472 109.753272 \n",
       "L 54.167615 108.406825 \n",
       "L 54.563159 107.082419 \n",
       "L 54.959083 105.780123 \n",
       "L 55.355366 104.499999 \n",
       "L 55.75199 103.242109 \n",
       "L 56.14893 102.006517 \n",
       "L 56.546167 100.793277 \n",
       "L 56.943682 99.602444 \n",
       "L 57.341452 98.434073 \n",
       "L 57.739456 97.288215 \n",
       "L 58.137675 96.164913 \n",
       "L 58.536086 95.064219 \n",
       "L 58.93467 93.986171 \n",
       "L 59.333404 92.930813 \n",
       "L 59.732268 91.898183 \n",
       "L 60.131242 90.888319 \n",
       "L 60.530306 89.901247 \n",
       "L 60.929437 88.937008 \n",
       "L 61.328615 87.995623 \n",
       "L 61.727819 87.077122 \n",
       "L 62.127029 86.181532 \n",
       "L 62.526223 85.308868 \n",
       "L 62.925381 84.459155 \n",
       "L 63.324482 83.632406 \n",
       "L 63.723507 82.828637 \n",
       "L 64.122433 82.04786 \n",
       "L 64.52124 81.290084 \n",
       "L 64.919909 80.555315 \n",
       "L 65.318418 79.84356 \n",
       "L 65.716748 79.15482 \n",
       "L 66.114878 78.489096 \n",
       "L 66.512787 77.846384 \n",
       "L 66.910456 77.22668 \n",
       "L 67.307865 76.629977 \n",
       "L 67.704992 76.056265 \n",
       "L 68.101819 75.505534 \n",
       "L 68.498325 74.977767 \n",
       "L 68.894491 74.472949 \n",
       "L 69.290297 73.991062 \n",
       "L 69.685723 73.532083 \n",
       "L 70.08075 73.095989 \n",
       "L 70.475358 72.682755 \n",
       "L 70.869528 72.292353 \n",
       "L 71.263241 71.924752 \n",
       "L 71.656477 71.57992 \n",
       "L 72.049218 71.257822 \n",
       "L 72.441444 70.958421 \n",
       "L 72.833137 70.681679 \n",
       "L 73.224278 70.427552 \n",
       "L 73.614849 70.195999 \n",
       "L 74.004831 69.986973 \n",
       "L 74.394205 69.800426 \n",
       "L 74.782954 69.636309 \n",
       "L 75.171059 69.49457 \n",
       "L 75.558503 69.375154 \n",
       "L 75.945267 69.278006 \n",
       "L 76.331334 69.203066 \n",
       "L 76.716687 69.150275 \n",
       "L 77.101307 69.11957 \n",
       "L 77.485179 69.110887 \n",
       "L 77.868283 69.124161 \n",
       "L 78.250604 69.159321 \n",
       "L 78.632125 69.2163 \n",
       "L 79.01283 69.295024 \n",
       "L 79.392701 69.395419 \n",
       "L 79.771721 69.517411 \n",
       "L 80.149876 69.66092 \n",
       "L 80.52715 69.825869 \n",
       "L 80.903524 70.012175 \n",
       "L 81.278986 70.219756 \n",
       "L 81.653518 70.448527 \n",
       "L 82.027106 70.698401 \n",
       "L 82.399735 70.969291 \n",
       "L 82.771387 71.261106 \n",
       "L 83.142052 71.573757 \n",
       "L 83.51171 71.907148 \n",
       "L 83.880351 72.261187 \n",
       "L 84.247957 72.635775 \n",
       "L 84.614516 73.030816 \n",
       "L 84.980014 73.446211 \n",
       "L 85.344435 73.881857 \n",
       "L 85.707768 74.337655 \n",
       "L 86.069998 74.813499 \n",
       "L 86.431112 75.309284 \n",
       "L 86.791098 75.824908 \n",
       "L 87.149941 76.360256 \n",
       "L 87.507629 76.915223 \n",
       "L 87.864152 77.489699 \n",
       "L 88.219493 78.083571 \n",
       "L 88.573643 78.696726 \n",
       "L 88.92659 79.329052 \n",
       "L 89.27832 79.98043 \n",
       "L 89.628823 80.650746 \n",
       "L 89.978089 81.339882 \n",
       "L 90.326103 82.047717 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 58.566252 120.082802 \n",
       "L 58.961016 118.622062 \n",
       "L 59.356168 117.183014 \n",
       "L 59.751687 115.765744 \n",
       "L 60.147551 114.370327 \n",
       "L 60.543743 112.996831 \n",
       "L 60.940238 111.645338 \n",
       "L 61.337017 110.315912 \n",
       "L 61.73406 109.008622 \n",
       "L 62.131345 107.72353 \n",
       "L 62.528852 106.460699 \n",
       "L 62.92656 105.220193 \n",
       "L 63.324447 104.002069 \n",
       "L 63.722494 102.806378 \n",
       "L 64.120679 101.633178 \n",
       "L 64.51898 100.482521 \n",
       "L 64.917379 99.354449 \n",
       "L 65.315853 98.249016 \n",
       "L 65.714382 97.166257 \n",
       "L 66.112943 96.106222 \n",
       "L 66.511518 95.068945 \n",
       "L 66.910084 94.054464 \n",
       "L 67.308623 93.062809 \n",
       "L 67.707111 92.094017 \n",
       "L 68.10553 91.148113 \n",
       "L 68.503858 90.225126 \n",
       "L 68.902073 89.325081 \n",
       "L 69.300156 88.447998 \n",
       "L 69.698086 87.593898 \n",
       "L 70.095843 86.762796 \n",
       "L 70.493406 85.954708 \n",
       "L 70.890754 85.169647 \n",
       "L 71.287867 84.407621 \n",
       "L 71.684726 83.668637 \n",
       "L 72.081309 82.952702 \n",
       "L 72.477597 82.259816 \n",
       "L 72.873568 81.589982 \n",
       "L 73.269205 80.943196 \n",
       "L 73.664486 80.319452 \n",
       "L 74.059391 79.718746 \n",
       "L 74.453901 79.141067 \n",
       "L 74.847996 78.586404 \n",
       "L 75.241656 78.054741 \n",
       "L 75.634862 77.546063 \n",
       "L 76.027595 77.060352 \n",
       "L 76.419835 76.597586 \n",
       "L 76.811562 76.15774 \n",
       "L 77.202759 75.740791 \n",
       "L 77.593405 75.34671 \n",
       "L 77.983482 74.975466 \n",
       "L 78.372972 74.627027 \n",
       "L 78.761855 74.301359 \n",
       "L 79.150112 73.998423 \n",
       "L 79.537727 73.718182 \n",
       "L 79.924679 73.460593 \n",
       "L 80.310952 73.225613 \n",
       "L 80.696527 73.013196 \n",
       "L 81.081386 72.823295 \n",
       "L 81.465512 72.655859 \n",
       "L 81.848886 72.510836 \n",
       "L 82.231491 72.388172 \n",
       "L 82.61331 72.28781 \n",
       "L 82.994326 72.209692 \n",
       "L 83.374522 72.153758 \n",
       "L 83.753881 72.119945 \n",
       "L 84.132386 72.108189 \n",
       "L 84.510019 72.118423 \n",
       "L 84.886767 72.150579 \n",
       "L 85.262611 72.204586 \n",
       "L 85.637535 72.280373 \n",
       "L 86.011525 72.377866 \n",
       "L 86.384563 72.496987 \n",
       "L 86.756634 72.63766 \n",
       "L 87.127724 72.799804 \n",
       "L 87.497816 72.983339 \n",
       "L 87.866895 73.188181 \n",
       "L 88.234947 73.414244 \n",
       "L 88.601956 73.661443 \n",
       "L 88.967909 73.92969 \n",
       "L 89.33279 74.218893 \n",
       "L 89.696587 74.528961 \n",
       "L 90.059283 74.859802 \n",
       "L 90.420866 75.211319 \n",
       "L 90.781321 75.583417 \n",
       "L 91.140635 75.975996 \n",
       "L 91.498797 76.388959 \n",
       "L 91.855789 76.822202 \n",
       "L 92.211602 77.275625 \n",
       "L 92.566222 77.749123 \n",
       "L 92.919636 78.242588 \n",
       "L 93.271833 78.755921 \n",
       "L 93.622798 79.289005 \n",
       "L 93.972522 79.841734 \n",
       "L 94.320992 80.413999 \n",
       "L 94.668196 81.005686 \n",
       "L 95.014122 81.61668 \n",
       "L 95.35876 82.246871 \n",
       "L 95.702098 82.896139 \n",
       "L 96.044126 83.564369 \n",
       "L 96.384833 84.251443 \n",
       "L 96.724208 84.95724 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 65.356543 121.185182 \n",
       "L 65.753953 119.716513 \n",
       "L 66.151634 118.269634 \n",
       "L 66.549565 116.844634 \n",
       "L 66.947725 115.441586 \n",
       "L 67.346094 114.060561 \n",
       "L 67.744649 112.701641 \n",
       "L 68.14337 111.36489 \n",
       "L 68.542236 110.05038 \n",
       "L 68.941226 108.758171 \n",
       "L 69.340321 107.488329 \n",
       "L 69.739496 106.240917 \n",
       "L 70.138733 105.015993 \n",
       "L 70.538011 103.813609 \n",
       "L 70.937306 102.633825 \n",
       "L 71.3366 101.476691 \n",
       "L 71.735871 100.34225 \n",
       "L 72.135098 99.230558 \n",
       "L 72.534261 98.141651 \n",
       "L 72.933337 97.075575 \n",
       "L 73.332306 96.032369 \n",
       "L 73.731147 95.012072 \n",
       "L 74.129841 94.014709 \n",
       "L 74.528365 93.040324 \n",
       "L 74.926699 92.088936 \n",
       "L 75.324823 91.16058 \n",
       "L 75.722714 90.255278 \n",
       "L 76.120355 89.37305 \n",
       "L 76.517722 88.51392 \n",
       "L 76.914796 87.6779 \n",
       "L 77.311557 86.865008 \n",
       "L 77.707984 86.075257 \n",
       "L 78.104057 85.308656 \n",
       "L 78.499756 84.56521 \n",
       "L 78.89506 83.844928 \n",
       "L 79.289951 83.147809 \n",
       "L 79.684406 82.473856 \n",
       "L 80.078408 81.823065 \n",
       "L 80.471936 81.195431 \n",
       "L 80.864969 80.590949 \n",
       "L 81.25749 80.009608 \n",
       "L 81.649477 79.451397 \n",
       "L 82.040912 78.916301 \n",
       "L 82.431777 78.404304 \n",
       "L 82.82205 77.915387 \n",
       "L 83.211713 77.449528 \n",
       "L 83.600748 77.006704 \n",
       "L 83.989136 76.586889 \n",
       "L 84.376857 76.190055 \n",
       "L 84.763893 75.816172 \n",
       "L 85.150226 75.465205 \n",
       "L 85.535838 75.137121 \n",
       "L 85.92071 74.831882 \n",
       "L 86.304824 74.549448 \n",
       "L 86.688163 74.289777 \n",
       "L 87.070707 74.052826 \n",
       "L 87.452442 73.838548 \n",
       "L 87.833347 73.646896 \n",
       "L 88.213406 73.477817 \n",
       "L 88.592602 73.331261 \n",
       "L 88.970917 73.207172 \n",
       "L 89.348336 73.105493 \n",
       "L 89.72484 73.026164 \n",
       "L 90.100414 72.969126 \n",
       "L 90.475042 72.934316 \n",
       "L 90.848706 72.921667 \n",
       "L 91.22139 72.931113 \n",
       "L 91.59308 72.962585 \n",
       "L 91.963758 73.016012 \n",
       "L 92.33341 73.091321 \n",
       "L 92.702021 73.188438 \n",
       "L 93.069573 73.307285 \n",
       "L 93.436053 73.447784 \n",
       "L 93.801447 73.609854 \n",
       "L 94.165738 73.793414 \n",
       "L 94.528913 73.998379 \n",
       "L 94.890957 74.224663 \n",
       "L 95.251855 74.47218 \n",
       "L 95.611596 74.740839 \n",
       "L 95.970163 75.03055 \n",
       "L 96.327545 75.341221 \n",
       "L 96.683725 75.672757 \n",
       "L 97.038694 76.025063 \n",
       "L 97.392435 76.398039 \n",
       "L 97.744937 76.791588 \n",
       "L 98.096188 77.205611 \n",
       "L 98.446174 77.640002 \n",
       "L 98.794883 78.094661 \n",
       "L 99.142304 78.569482 \n",
       "L 99.488424 79.064356 \n",
       "L 99.833233 79.579182 \n",
       "L 100.176717 80.113844 \n",
       "L 100.518866 80.668234 \n",
       "L 100.859669 81.24224 \n",
       "L 101.199115 81.83575 \n",
       "L 101.537192 82.448647 \n",
       "L 101.873891 83.08082 \n",
       "L 102.209201 83.732148 \n",
       "L 102.543112 84.402514 \n",
       "L 102.875615 85.0918 \n",
       "L 103.206697 85.799883 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 72.210813 120.090746 \n",
       "L 72.61161 118.611293 \n",
       "L 73.01256 117.153808 \n",
       "L 73.413639 115.718379 \n",
       "L 73.814828 114.305082 \n",
       "L 74.216107 112.913989 \n",
       "L 74.617451 111.545182 \n",
       "L 75.018842 110.198727 \n",
       "L 75.420256 108.874696 \n",
       "L 75.821674 107.573151 \n",
       "L 76.223075 106.294158 \n",
       "L 76.624435 105.037781 \n",
       "L 77.025735 103.804079 \n",
       "L 77.426954 102.593107 \n",
       "L 77.828068 101.404921 \n",
       "L 78.229058 100.239576 \n",
       "L 78.629904 99.097114 \n",
       "L 79.030583 97.977591 \n",
       "L 79.431074 96.881044 \n",
       "L 79.831356 95.807523 \n",
       "L 80.231408 94.757063 \n",
       "L 80.631209 93.729703 \n",
       "L 81.030739 92.725473 \n",
       "L 81.429976 91.744413 \n",
       "L 81.8289 90.786546 \n",
       "L 82.22749 89.851904 \n",
       "L 82.625724 88.940513 \n",
       "L 83.023583 88.05239 \n",
       "L 83.421046 87.18756 \n",
       "L 83.818092 86.346037 \n",
       "L 84.214702 85.527837 \n",
       "L 84.610853 84.732974 \n",
       "L 85.006527 83.961456 \n",
       "L 85.401704 83.21329 \n",
       "L 85.796363 82.488483 \n",
       "L 86.190484 81.787036 \n",
       "L 86.584047 81.108951 \n",
       "L 86.977034 80.454223 \n",
       "L 87.369423 79.822848 \n",
       "L 87.761196 79.21482 \n",
       "L 88.152332 78.630129 \n",
       "L 88.542813 78.068763 \n",
       "L 88.93262 77.530706 \n",
       "L 89.321733 77.015942 \n",
       "L 89.710133 76.524452 \n",
       "L 90.097801 76.056214 \n",
       "L 90.484719 75.611203 \n",
       "L 90.870869 75.189395 \n",
       "L 91.256231 74.790759 \n",
       "L 91.640787 74.415265 \n",
       "L 92.02452 74.062879 \n",
       "L 92.407411 73.733565 \n",
       "L 92.789442 73.427286 \n",
       "L 93.170596 73.144002 \n",
       "L 93.550854 72.883669 \n",
       "L 93.9302 72.646242 \n",
       "L 94.308617 72.431676 \n",
       "L 94.686086 72.239921 \n",
       "L 95.062592 72.070925 \n",
       "L 95.438116 71.924635 \n",
       "L 95.812643 71.800995 \n",
       "L 96.186156 71.699948 \n",
       "L 96.558639 71.621432 \n",
       "L 96.930076 71.565388 \n",
       "L 97.30045 71.531749 \n",
       "L 97.669746 71.520451 \n",
       "L 98.037948 71.531425 \n",
       "L 98.405041 71.5646 \n",
       "L 98.771009 71.619905 \n",
       "L 99.135838 71.697267 \n",
       "L 99.499512 71.796607 \n",
       "L 99.862017 71.91785 \n",
       "L 100.223338 72.060914 \n",
       "L 100.583461 72.225718 \n",
       "L 100.942371 72.41218 \n",
       "L 101.300055 72.620212 \n",
       "L 101.656499 72.849729 \n",
       "L 102.011689 73.100641 \n",
       "L 102.365613 73.372858 \n",
       "L 102.718255 73.666287 \n",
       "L 103.069605 73.980835 \n",
       "L 103.419648 74.316406 \n",
       "L 103.768372 74.672902 \n",
       "L 104.115765 75.050225 \n",
       "L 104.461814 75.448273 \n",
       "L 104.806508 75.866946 \n",
       "L 105.149834 76.306138 \n",
       "L 105.491781 76.765747 \n",
       "L 105.832338 77.245663 \n",
       "L 106.171493 77.745779 \n",
       "L 106.509236 78.265991 \n",
       "L 106.845554 78.80618 \n",
       "L 107.180439 79.366238 \n",
       "L 107.513879 79.946052 \n",
       "L 107.845864 80.545506 \n",
       "L 108.176384 81.164483 \n",
       "L 108.50543 81.80287 \n",
       "L 108.832991 82.460543 \n",
       "L 109.159058 83.137385 \n",
       "L 109.483624 83.833276 \n",
       "L 109.806676 84.548092 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 79.167147 116.745165 \n",
       "L 79.572108 115.251949 \n",
       "L 79.977101 113.78096 \n",
       "L 80.3821 112.332288 \n",
       "L 80.787086 110.90601 \n",
       "L 81.192038 109.502199 \n",
       "L 81.596932 108.120939 \n",
       "L 82.001748 106.762297 \n",
       "L 82.406463 105.426347 \n",
       "L 82.811056 104.11315 \n",
       "L 83.215508 102.822776 \n",
       "L 83.619793 101.555288 \n",
       "L 84.023891 100.310747 \n",
       "L 84.427783 99.089208 \n",
       "L 84.831443 97.890731 \n",
       "L 85.234853 96.715367 \n",
       "L 85.637991 95.563164 \n",
       "L 86.040834 94.434174 \n",
       "L 86.443364 93.328438 \n",
       "L 86.845555 92.246005 \n",
       "L 87.247389 91.186912 \n",
       "L 87.648843 90.151198 \n",
       "L 88.049899 89.138893 \n",
       "L 88.450533 88.150038 \n",
       "L 88.850726 87.184657 \n",
       "L 89.250455 86.242781 \n",
       "L 89.6497 85.324437 \n",
       "L 90.048441 84.429643 \n",
       "L 90.446656 83.558422 \n",
       "L 90.844327 82.710791 \n",
       "L 91.241431 81.886764 \n",
       "L 91.637948 81.086356 \n",
       "L 92.033859 80.309574 \n",
       "L 92.429143 79.556427 \n",
       "L 92.82378 78.82692 \n",
       "L 93.21775 78.121055 \n",
       "L 93.611033 77.438832 \n",
       "L 94.00361 76.780248 \n",
       "L 94.395461 76.145297 \n",
       "L 94.786566 75.533975 \n",
       "L 95.176906 74.946268 \n",
       "L 95.566462 74.382165 \n",
       "L 95.955215 73.841651 \n",
       "L 96.343146 73.324708 \n",
       "L 96.730235 72.831317 \n",
       "L 97.116465 72.361454 \n",
       "L 97.501817 71.915096 \n",
       "L 97.886272 71.492216 \n",
       "L 98.269812 71.092782 \n",
       "L 98.652419 70.716765 \n",
       "L 99.034075 70.364128 \n",
       "L 99.414762 70.034836 \n",
       "L 99.794463 69.72885 \n",
       "L 100.17316 69.446128 \n",
       "L 100.550836 69.186626 \n",
       "L 100.927474 68.950299 \n",
       "L 101.303057 68.737098 \n",
       "L 101.677567 68.546974 \n",
       "L 102.050988 68.379874 \n",
       "L 102.423304 68.235741 \n",
       "L 102.794499 68.114521 \n",
       "L 103.164556 68.016153 \n",
       "L 103.533459 67.940576 \n",
       "L 103.901194 67.887727 \n",
       "L 104.267743 67.857539 \n",
       "L 104.633093 67.849946 \n",
       "L 104.997226 67.864878 \n",
       "L 105.36013 67.902262 \n",
       "L 105.721788 67.962025 \n",
       "L 106.082186 68.044092 \n",
       "L 106.44131 68.148384 \n",
       "L 106.799146 68.274822 \n",
       "L 107.155678 68.423324 \n",
       "L 107.510895 68.593807 \n",
       "L 107.864782 68.786185 \n",
       "L 108.217325 69.000371 \n",
       "L 108.568512 69.236276 \n",
       "L 108.918328 69.493809 \n",
       "L 109.266764 69.772878 \n",
       "L 109.613803 70.073388 \n",
       "L 109.959435 70.395243 \n",
       "L 110.303647 70.738347 \n",
       "L 110.646428 71.102598 \n",
       "L 110.987765 71.487896 \n",
       "L 111.327646 71.894138 \n",
       "L 111.666062 72.321221 \n",
       "L 112.002999 72.769037 \n",
       "L 112.338447 73.237481 \n",
       "L 112.672396 73.726443 \n",
       "L 113.004835 74.235812 \n",
       "L 113.335755 74.765482 \n",
       "L 113.665143 75.315331 \n",
       "L 113.992991 75.88525 \n",
       "L 114.31929 76.475123 \n",
       "L 114.644028 77.084831 \n",
       "L 114.967197 77.714256 \n",
       "L 115.28879 78.363282 \n",
       "L 115.608795 79.031783 \n",
       "L 115.927205 79.719639 \n",
       "L 116.244011 80.426727 \n",
       "L 116.559205 81.152921 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 86.266001 111.06794 \n",
       "L 86.675953 109.55782 \n",
       "L 87.085808 108.070273 \n",
       "L 87.495543 106.605391 \n",
       "L 87.905136 105.163252 \n",
       "L 88.314566 103.743932 \n",
       "L 88.723809 102.347516 \n",
       "L 89.132844 100.974072 \n",
       "L 89.541649 99.623676 \n",
       "L 89.950202 98.296392 \n",
       "L 90.358481 96.99229 \n",
       "L 90.766463 95.711434 \n",
       "L 91.174126 94.453888 \n",
       "L 91.581451 93.219705 \n",
       "L 91.988412 92.008948 \n",
       "L 92.394989 90.821672 \n",
       "L 92.801162 89.65792 \n",
       "L 93.206906 88.51775 \n",
       "L 93.612202 87.401202 \n",
       "L 94.017026 86.308325 \n",
       "L 94.421359 85.239157 \n",
       "L 94.825177 84.193739 \n",
       "L 95.228462 83.172101 \n",
       "L 95.63119 82.174284 \n",
       "L 96.033342 81.200313 \n",
       "L 96.434894 80.250219 \n",
       "L 96.835827 79.324028 \n",
       "L 97.23612 78.421761 \n",
       "L 97.635751 77.543441 \n",
       "L 98.034701 76.689083 \n",
       "L 98.432948 75.858701 \n",
       "L 98.830472 75.052312 \n",
       "L 99.227253 74.269922 \n",
       "L 99.623271 73.51154 \n",
       "L 100.018505 72.777171 \n",
       "L 100.412937 72.066814 \n",
       "L 100.806544 71.380474 \n",
       "L 101.199308 70.718143 \n",
       "L 101.59121 70.079817 \n",
       "L 101.98223 69.465487 \n",
       "L 102.372347 68.875146 \n",
       "L 102.761545 68.308776 \n",
       "L 103.149802 67.766364 \n",
       "L 103.537102 67.247889 \n",
       "L 103.923424 66.753335 \n",
       "L 104.308749 66.282674 \n",
       "L 104.693061 65.835883 \n",
       "L 105.076341 65.412932 \n",
       "L 105.458569 65.013793 \n",
       "L 105.839729 64.638428 \n",
       "L 106.219803 64.286806 \n",
       "L 106.598774 63.958887 \n",
       "L 106.976622 63.654631 \n",
       "L 107.353333 63.373995 \n",
       "L 107.728888 63.116934 \n",
       "L 108.10327 62.883401 \n",
       "L 108.476465 62.673345 \n",
       "L 108.848452 62.486715 \n",
       "L 109.219218 62.323458 \n",
       "L 109.588746 62.183515 \n",
       "L 109.95702 62.066829 \n",
       "L 110.324025 61.973336 \n",
       "L 110.689743 61.902977 \n",
       "L 111.054162 61.855682 \n",
       "L 111.417265 61.831387 \n",
       "L 111.779037 61.830021 \n",
       "L 112.139463 61.851511 \n",
       "L 112.49853 61.895785 \n",
       "L 112.856222 61.962765 \n",
       "L 113.212526 62.052374 \n",
       "L 113.567427 62.164534 \n",
       "L 113.920912 62.299159 \n",
       "L 114.272967 62.456168 \n",
       "L 114.623579 62.635473 \n",
       "L 114.972735 62.836988 \n",
       "L 115.320422 63.060622 \n",
       "L 115.666628 63.306284 \n",
       "L 116.011339 63.57388 \n",
       "L 116.354545 63.863316 \n",
       "L 116.696232 64.174493 \n",
       "L 117.036389 64.507314 \n",
       "L 117.375004 64.861678 \n",
       "L 117.712066 65.237483 \n",
       "L 118.047564 65.634625 \n",
       "L 118.381486 66.052997 \n",
       "L 118.713823 66.492495 \n",
       "L 119.044563 66.953007 \n",
       "L 119.373696 67.434426 \n",
       "L 119.701212 67.936638 \n",
       "L 120.027102 68.459529 \n",
       "L 120.351357 69.00299 \n",
       "L 120.673965 69.566897 \n",
       "L 120.994919 70.151137 \n",
       "L 121.31421 70.755592 \n",
       "L 121.631827 71.380139 \n",
       "L 121.947764 72.024657 \n",
       "L 122.262012 72.689026 \n",
       "L 122.574562 73.373118 \n",
       "L 122.885407 74.07681 \n",
       "L 123.19454 74.799976 \n",
       "L 123.501952 75.542486 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 93.551058 102.94972 \n",
       "L 93.966883 101.419348 \n",
       "L 94.382478 99.911988 \n",
       "L 94.797818 98.427736 \n",
       "L 95.212882 96.966671 \n",
       "L 95.627647 95.528871 \n",
       "L 96.042089 94.114424 \n",
       "L 96.456186 92.7234 \n",
       "L 96.869916 91.355875 \n",
       "L 97.283256 90.011917 \n",
       "L 97.696184 88.691595 \n",
       "L 98.108676 87.394978 \n",
       "L 98.52071 86.122128 \n",
       "L 98.932266 84.873103 \n",
       "L 99.343318 83.647965 \n",
       "L 99.753845 82.44677 \n",
       "L 100.163827 81.269564 \n",
       "L 100.573239 80.116405 \n",
       "L 100.982061 78.987334 \n",
       "L 101.390269 77.8824 \n",
       "L 101.797842 76.801644 \n",
       "L 102.204759 75.745106 \n",
       "L 102.610999 74.712818 \n",
       "L 103.016538 73.70482 \n",
       "L 103.421357 72.721139 \n",
       "L 103.825433 71.761806 \n",
       "L 104.228744 70.826846 \n",
       "L 104.631272 69.916282 \n",
       "L 105.032992 69.030137 \n",
       "L 105.433887 68.168425 \n",
       "L 105.833934 67.331161 \n",
       "L 106.233112 66.518361 \n",
       "L 106.631402 65.730031 \n",
       "L 107.028783 64.96618 \n",
       "L 107.425234 64.226813 \n",
       "L 107.820736 63.51193 \n",
       "L 108.215269 62.821529 \n",
       "L 108.608813 62.155609 \n",
       "L 109.001348 61.514162 \n",
       "L 109.392855 60.897179 \n",
       "L 109.783314 60.30465 \n",
       "L 110.172707 59.73656 \n",
       "L 110.561014 59.192891 \n",
       "L 110.948216 58.673626 \n",
       "L 111.334296 58.17874 \n",
       "L 111.719233 57.70821 \n",
       "L 112.103011 57.262009 \n",
       "L 112.48561 56.840108 \n",
       "L 112.867014 56.442475 \n",
       "L 113.247203 56.069072 \n",
       "L 113.626162 55.719865 \n",
       "L 114.003871 55.394814 \n",
       "L 114.380315 55.093877 \n",
       "L 114.755475 54.817007 \n",
       "L 115.129336 54.564159 \n",
       "L 115.50188 54.335283 \n",
       "L 115.873092 54.130328 \n",
       "L 116.242954 53.94924 \n",
       "L 116.611452 53.79196 \n",
       "L 116.978568 53.658431 \n",
       "L 117.344288 53.548592 \n",
       "L 117.708596 53.462378 \n",
       "L 118.071477 53.399726 \n",
       "L 118.432916 53.360565 \n",
       "L 118.792898 53.344827 \n",
       "L 119.151409 53.352439 \n",
       "L 119.508434 53.383325 \n",
       "L 119.86396 53.437409 \n",
       "L 120.217972 53.514613 \n",
       "L 120.570457 53.614857 \n",
       "L 120.921401 53.738056 \n",
       "L 121.27079 53.884125 \n",
       "L 121.618613 54.052978 \n",
       "L 121.964857 54.244526 \n",
       "L 122.309508 54.458676 \n",
       "L 122.652554 54.695338 \n",
       "L 122.993984 54.954415 \n",
       "L 123.333785 55.235811 \n",
       "L 123.671947 55.539428 \n",
       "L 124.008457 55.865164 \n",
       "L 124.343305 56.212918 \n",
       "L 124.676478 56.582587 \n",
       "L 125.007968 56.974063 \n",
       "L 125.337763 57.387241 \n",
       "L 125.665852 57.822009 \n",
       "L 125.992227 58.278259 \n",
       "L 126.316876 58.755876 \n",
       "L 126.639791 59.25475 \n",
       "L 126.960962 59.774762 \n",
       "L 127.280381 60.315795 \n",
       "L 127.598039 60.877736 \n",
       "L 127.913925 61.460457 \n",
       "L 128.228033 62.063841 \n",
       "L 128.540355 62.687766 \n",
       "L 128.850882 63.332106 \n",
       "L 129.159606 63.996735 \n",
       "L 129.466521 64.68153 \n",
       "L 129.771618 65.386359 \n",
       "L 130.074891 66.111094 \n",
       "L 130.376334 66.855605 \n",
       "L 130.675939 67.619758 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 24.318635 81.868935 \n",
       "L 25.025641 83.182889 \n",
       "L 25.731159 84.473354 \n",
       "L 26.435225 85.740419 \n",
       "L 27.137878 86.984178 \n",
       "L 27.839158 88.204725 \n",
       "L 28.5391 89.402139 \n",
       "L 29.237744 90.57651 \n",
       "L 29.935126 91.727916 \n",
       "L 30.631285 92.856443 \n",
       "L 31.326259 93.962164 \n",
       "L 32.020082 95.045152 \n",
       "L 32.712793 96.105481 \n",
       "L 33.40443 97.143223 \n",
       "L 34.095027 98.158442 \n",
       "L 34.784621 99.151201 \n",
       "L 35.47325 100.121569 \n",
       "L 36.160949 101.069599 \n",
       "L 36.847755 101.995355 \n",
       "L 37.533702 102.898888 \n",
       "L 38.218828 103.780253 \n",
       "L 38.903167 104.639499 \n",
       "L 39.586759 105.476678 \n",
       "L 40.269634 106.291834 \n",
       "L 40.951832 107.08501 \n",
       "L 41.633387 107.856249 \n",
       "L 42.314332 108.60559 \n",
       "L 42.994705 109.33307 \n",
       "L 43.674541 110.038721 \n",
       "L 44.353876 110.722579 \n",
       "L 45.032744 111.384675 \n",
       "L 45.711179 112.025033 \n",
       "L 46.389219 112.643682 \n",
       "L 47.066898 113.240644 \n",
       "L 47.74425 113.815941 \n",
       "L 48.421312 114.36959 \n",
       "L 49.098118 114.901611 \n",
       "L 49.774704 115.412016 \n",
       "L 50.451104 115.900818 \n",
       "L 51.127353 116.368029 \n",
       "L 51.803488 116.813651 \n",
       "L 52.479542 117.237696 \n",
       "L 53.155551 117.640163 \n",
       "L 53.831551 118.021055 \n",
       "L 54.507577 118.380372 \n",
       "L 55.183663 118.718108 \n",
       "L 55.859846 119.034257 \n",
       "L 56.536161 119.328813 \n",
       "L 57.212643 119.601764 \n",
       "L 57.889328 119.853099 \n",
       "L 58.566252 120.082802 \n",
       "L 59.24345 120.290857 \n",
       "L 59.920959 120.477242 \n",
       "L 60.598814 120.64194 \n",
       "L 61.277052 120.784923 \n",
       "L 61.955708 120.906165 \n",
       "L 62.63482 121.005638 \n",
       "L 63.314422 121.08331 \n",
       "L 63.994552 121.13915 \n",
       "L 64.675247 121.17312 \n",
       "L 65.356543 121.185182 \n",
       "L 66.038477 121.175298 \n",
       "L 66.721087 121.14342 \n",
       "L 67.404411 121.089506 \n",
       "L 68.088485 121.013508 \n",
       "L 68.773347 120.915374 \n",
       "L 69.459035 120.795054 \n",
       "L 70.145587 120.65249 \n",
       "L 70.833041 120.487625 \n",
       "L 71.521438 120.300398 \n",
       "L 72.210813 120.090746 \n",
       "L 72.901207 119.858606 \n",
       "L 73.592658 119.603907 \n",
       "L 74.285208 119.326579 \n",
       "L 74.978894 119.026549 \n",
       "L 75.673757 118.703739 \n",
       "L 76.369837 118.358074 \n",
       "L 77.067175 117.989468 \n",
       "L 77.765813 117.597841 \n",
       "L 78.465788 117.183103 \n",
       "L 79.167147 116.745165 \n",
       "L 79.869926 116.283934 \n",
       "L 80.574171 115.799315 \n",
       "L 81.279922 115.291208 \n",
       "L 81.987221 114.759515 \n",
       "L 82.696116 114.204127 \n",
       "L 83.406645 113.624942 \n",
       "L 84.118853 113.021844 \n",
       "L 84.832785 112.394724 \n",
       "L 85.548485 111.743465 \n",
       "L 86.266001 111.06794 \n",
       "L 86.985373 110.368036 \n",
       "L 87.70665 109.643622 \n",
       "L 88.429879 108.894568 \n",
       "L 89.155103 108.120743 \n",
       "L 89.882371 107.322012 \n",
       "L 90.611733 106.498229 \n",
       "L 91.343233 105.649259 \n",
       "L 92.07692 104.774951 \n",
       "L 92.812847 103.875155 \n",
       "L 93.551058 102.94972 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 28.279656 68.264617 \n",
       "L 28.985001 69.583979 \n",
       "L 29.688929 70.879613 \n",
       "L 30.391474 72.151607 \n",
       "L 31.092677 73.400058 \n",
       "L 31.792577 74.62506 \n",
       "L 32.491208 75.82669 \n",
       "L 33.188611 77.005041 \n",
       "L 33.884822 78.160191 \n",
       "L 34.579879 79.292226 \n",
       "L 35.27382 80.401219 \n",
       "L 35.96668 81.487244 \n",
       "L 36.658496 82.550375 \n",
       "L 37.349307 83.590684 \n",
       "L 38.039147 84.608235 \n",
       "L 38.728053 85.603093 \n",
       "L 39.416063 86.575325 \n",
       "L 40.103211 87.524986 \n",
       "L 40.789535 88.45214 \n",
       "L 41.475069 89.356836 \n",
       "L 42.159849 90.239131 \n",
       "L 42.843912 91.099074 \n",
       "L 43.527296 91.936718 \n",
       "L 44.210031 92.752104 \n",
       "L 44.892158 93.545279 \n",
       "L 45.573709 94.316283 \n",
       "L 46.25472 95.065154 \n",
       "L 46.935228 95.791932 \n",
       "L 47.615265 96.496649 \n",
       "L 48.294871 97.179339 \n",
       "L 48.974078 97.840034 \n",
       "L 49.652921 98.478756 \n",
       "L 50.331436 99.095536 \n",
       "L 51.00966 99.690396 \n",
       "L 51.687626 100.263355 \n",
       "L 52.36537 100.814434 \n",
       "L 53.042927 101.343647 \n",
       "L 53.720332 101.851011 \n",
       "L 54.397621 102.336537 \n",
       "L 55.074829 102.800235 \n",
       "L 55.75199 103.242109 \n",
       "L 56.42914 103.662169 \n",
       "L 57.106315 104.060415 \n",
       "L 57.78355 104.43685 \n",
       "L 58.46088 104.791468 \n",
       "L 59.13834 105.12427 \n",
       "L 59.815968 105.435246 \n",
       "L 60.493797 105.724389 \n",
       "L 61.171863 105.991687 \n",
       "L 61.850203 106.23713 \n",
       "L 62.528852 106.460699 \n",
       "L 63.207847 106.662378 \n",
       "L 63.887223 106.842146 \n",
       "L 64.567016 106.999982 \n",
       "L 65.247263 107.135861 \n",
       "L 65.928 107.249754 \n",
       "L 66.609265 107.341632 \n",
       "L 67.291091 107.411465 \n",
       "L 67.973519 107.459216 \n",
       "L 68.656583 107.484851 \n",
       "L 69.340321 107.488329 \n",
       "L 70.02477 107.46961 \n",
       "L 70.709969 107.428648 \n",
       "L 71.395954 107.365399 \n",
       "L 72.082764 107.279812 \n",
       "L 72.770436 107.171837 \n",
       "L 73.459008 107.04142 \n",
       "L 74.14852 106.888502 \n",
       "L 74.839008 106.71303 \n",
       "L 75.530514 106.514936 \n",
       "L 76.223075 106.294158 \n",
       "L 76.91673 106.05063 \n",
       "L 77.611519 105.784283 \n",
       "L 78.307483 105.495044 \n",
       "L 79.004661 105.182838 \n",
       "L 79.703094 104.847588 \n",
       "L 80.402821 104.489214 \n",
       "L 81.103884 104.107634 \n",
       "L 81.806327 103.702759 \n",
       "L 82.510186 103.274504 \n",
       "L 83.215508 102.822776 \n",
       "L 83.92233 102.34748 \n",
       "L 84.6307 101.84852 \n",
       "L 85.340655 101.325797 \n",
       "L 86.052242 100.779206 \n",
       "L 86.765505 100.208641 \n",
       "L 87.480485 99.613995 \n",
       "L 88.197227 98.995153 \n",
       "L 88.915777 98.352002 \n",
       "L 89.636179 97.684423 \n",
       "L 90.358481 96.99229 \n",
       "L 91.082725 96.275485 \n",
       "L 91.808959 95.533877 \n",
       "L 92.537231 94.767333 \n",
       "L 93.267586 93.97572 \n",
       "L 94.000072 93.1589 \n",
       "L 94.73474 92.316728 \n",
       "L 95.471634 91.449063 \n",
       "L 96.210806 90.555755 \n",
       "L 96.952308 89.63665 \n",
       "L 97.696184 88.691595 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 32.320305 56.954651 \n",
       "L 33.022734 58.276188 \n",
       "L 33.72382 59.573842 \n",
       "L 34.423599 60.847702 \n",
       "L 35.122109 62.097864 \n",
       "L 35.819392 63.32442 \n",
       "L 36.515479 64.527451 \n",
       "L 37.210411 65.707048 \n",
       "L 37.904225 66.86329 \n",
       "L 38.596958 67.996261 \n",
       "L 39.288648 69.106036 \n",
       "L 39.979329 70.192689 \n",
       "L 40.669039 71.256293 \n",
       "L 41.357817 72.29692 \n",
       "L 42.045695 73.314635 \n",
       "L 42.73271 74.309502 \n",
       "L 43.418902 75.281589 \n",
       "L 44.104303 76.23095 \n",
       "L 44.788951 77.157649 \n",
       "L 45.47288 78.061736 \n",
       "L 46.156126 78.943266 \n",
       "L 46.838726 79.802289 \n",
       "L 47.520716 80.638859 \n",
       "L 48.202129 81.453014 \n",
       "L 48.883004 82.244804 \n",
       "L 49.563373 83.014267 \n",
       "L 50.243271 83.761442 \n",
       "L 50.922736 84.486368 \n",
       "L 51.601802 85.189076 \n",
       "L 52.280504 85.869603 \n",
       "L 52.958878 86.527975 \n",
       "L 53.636957 87.16422 \n",
       "L 54.314778 87.778365 \n",
       "L 54.992377 88.370433 \n",
       "L 55.669787 88.940443 \n",
       "L 56.347045 89.488416 \n",
       "L 57.024184 90.014365 \n",
       "L 57.701241 90.518306 \n",
       "L 58.378251 91.000251 \n",
       "L 59.055248 91.460207 \n",
       "L 59.732268 91.898183 \n",
       "L 60.409347 92.314184 \n",
       "L 61.086518 92.708212 \n",
       "L 61.763819 93.080267 \n",
       "L 62.441284 93.430347 \n",
       "L 63.118949 93.758448 \n",
       "L 63.796849 94.064562 \n",
       "L 64.475021 94.348681 \n",
       "L 65.153499 94.610794 \n",
       "L 65.832319 94.850887 \n",
       "L 66.511518 95.068945 \n",
       "L 67.191131 95.264948 \n",
       "L 67.871195 95.438877 \n",
       "L 68.551746 95.590708 \n",
       "L 69.23282 95.720417 \n",
       "L 69.914454 95.827975 \n",
       "L 70.596685 95.913352 \n",
       "L 71.279548 95.976516 \n",
       "L 71.963081 96.017432 \n",
       "L 72.647321 96.036063 \n",
       "L 73.332306 96.032369 \n",
       "L 74.018071 96.006309 \n",
       "L 74.704657 95.957837 \n",
       "L 75.392099 95.886905 \n",
       "L 76.080437 95.793466 \n",
       "L 76.769708 95.677466 \n",
       "L 77.459949 95.538852 \n",
       "L 78.151202 95.377565 \n",
       "L 78.843502 95.193547 \n",
       "L 79.536891 94.986734 \n",
       "L 80.231408 94.757063 \n",
       "L 80.92709 94.504465 \n",
       "L 81.623979 94.22887 \n",
       "L 82.322115 93.930206 \n",
       "L 83.021537 93.608396 \n",
       "L 83.722287 93.263363 \n",
       "L 84.424405 92.895026 \n",
       "L 85.127931 92.503301 \n",
       "L 85.832911 92.088099 \n",
       "L 86.539382 91.649335 \n",
       "L 87.247389 91.186912 \n",
       "L 87.956971 90.700738 \n",
       "L 88.668175 90.190714 \n",
       "L 89.381039 89.656739 \n",
       "L 90.095611 89.09871 \n",
       "L 90.811934 88.516516 \n",
       "L 91.53005 87.910053 \n",
       "L 92.250005 87.279202 \n",
       "L 92.971844 86.62385 \n",
       "L 93.695613 85.943878 \n",
       "L 94.421359 85.239157 \n",
       "L 95.149124 84.50957 \n",
       "L 95.878957 83.754982 \n",
       "L 96.610908 82.97526 \n",
       "L 97.34502 82.170271 \n",
       "L 98.081342 81.339876 \n",
       "L 98.819927 80.483926 \n",
       "L 99.560818 79.602281 \n",
       "L 100.304066 78.694789 \n",
       "L 101.049726 77.761295 \n",
       "L 101.797842 76.801644 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 36.419139 47.985381 \n",
       "L 37.117426 49.305848 \n",
       "L 37.81445 50.602363 \n",
       "L 38.510244 51.875014 \n",
       "L 39.204849 53.123895 \n",
       "L 39.898303 54.3491 \n",
       "L 40.590639 55.55071 \n",
       "L 41.281897 56.728813 \n",
       "L 41.972113 57.883489 \n",
       "L 42.661324 59.014823 \n",
       "L 43.349569 60.122888 \n",
       "L 44.036879 61.207759 \n",
       "L 44.723294 62.269507 \n",
       "L 45.408851 63.308206 \n",
       "L 46.093582 64.323918 \n",
       "L 46.777525 65.316709 \n",
       "L 47.460718 66.286646 \n",
       "L 48.143193 67.233782 \n",
       "L 48.824989 68.158182 \n",
       "L 49.506137 69.059894 \n",
       "L 50.186676 69.938975 \n",
       "L 50.866641 70.795473 \n",
       "L 51.546068 71.629441 \n",
       "L 52.224989 72.440919 \n",
       "L 52.903443 73.229955 \n",
       "L 53.581463 73.996587 \n",
       "L 54.259083 74.740854 \n",
       "L 54.936341 75.462795 \n",
       "L 55.613269 76.162441 \n",
       "L 56.289904 76.839826 \n",
       "L 56.966281 77.49498 \n",
       "L 57.642433 78.127928 \n",
       "L 58.318396 78.738697 \n",
       "L 58.994207 79.327309 \n",
       "L 59.669897 79.893785 \n",
       "L 60.345505 80.438143 \n",
       "L 61.021063 80.960398 \n",
       "L 61.696607 81.460566 \n",
       "L 62.372173 81.938655 \n",
       "L 63.047794 82.394677 \n",
       "L 63.723507 82.828637 \n",
       "L 64.399345 83.240539 \n",
       "L 65.075346 83.630387 \n",
       "L 65.751543 83.998181 \n",
       "L 66.427972 84.343917 \n",
       "L 67.104668 84.667591 \n",
       "L 67.781668 84.969196 \n",
       "L 68.459006 85.248723 \n",
       "L 69.136717 85.50616 \n",
       "L 69.814839 85.741494 \n",
       "L 70.493406 85.954708 \n",
       "L 71.172455 86.145784 \n",
       "L 71.852021 86.3147 \n",
       "L 72.532141 86.461433 \n",
       "L 73.212852 86.585959 \n",
       "L 73.894188 86.688249 \n",
       "L 74.57619 86.768272 \n",
       "L 75.258889 86.825996 \n",
       "L 75.942326 86.861385 \n",
       "L 76.626536 86.874403 \n",
       "L 77.311557 86.865008 \n",
       "L 77.997427 86.83316 \n",
       "L 78.684183 86.778811 \n",
       "L 79.371862 86.701917 \n",
       "L 80.060504 86.602425 \n",
       "L 80.750146 86.480285 \n",
       "L 81.440824 86.335441 \n",
       "L 82.132581 86.167835 \n",
       "L 82.825453 85.977408 \n",
       "L 83.51948 85.764097 \n",
       "L 84.214702 85.527837 \n",
       "L 84.911156 85.268559 \n",
       "L 85.608884 84.986195 \n",
       "L 86.307926 84.680669 \n",
       "L 87.008322 84.351906 \n",
       "L 87.710113 83.999829 \n",
       "L 88.413338 83.624354 \n",
       "L 89.118041 83.225399 \n",
       "L 89.824264 82.802875 \n",
       "L 90.532045 82.356695 \n",
       "L 91.241431 81.886764 \n",
       "L 91.95246 81.392987 \n",
       "L 92.665179 80.875266 \n",
       "L 93.379626 80.333499 \n",
       "L 94.095849 79.767583 \n",
       "L 94.813892 79.177408 \n",
       "L 95.533797 78.562868 \n",
       "L 96.255609 77.923843 \n",
       "L 96.979374 77.260223 \n",
       "L 97.705138 76.571885 \n",
       "L 98.432948 75.858701 \n",
       "L 99.162847 75.120554 \n",
       "L 99.894885 74.357309 \n",
       "L 100.629109 73.568832 \n",
       "L 101.365565 72.754989 \n",
       "L 102.104301 71.915642 \n",
       "L 102.84537 71.050641 \n",
       "L 103.588816 70.159847 \n",
       "L 104.334692 69.243105 \n",
       "L 105.083049 68.300261 \n",
       "L 105.833934 67.331161 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 40.554333 41.378444 \n",
       "L 41.247296 42.694628 \n",
       "L 41.939079 43.986875 \n",
       "L 42.629714 45.255272 \n",
       "L 43.319239 46.499913 \n",
       "L 44.007694 47.720892 \n",
       "L 44.695112 48.918287 \n",
       "L 45.38153 50.092187 \n",
       "L 46.066985 51.242671 \n",
       "L 46.751513 52.369823 \n",
       "L 47.435153 53.473716 \n",
       "L 48.117935 54.554424 \n",
       "L 48.799899 55.612018 \n",
       "L 49.481081 56.64657 \n",
       "L 50.161513 57.658144 \n",
       "L 50.841233 58.646802 \n",
       "L 51.520278 59.612613 \n",
       "L 52.198679 60.555629 \n",
       "L 52.876475 61.475912 \n",
       "L 53.553697 62.373515 \n",
       "L 54.230384 63.248489 \n",
       "L 54.906568 64.100884 \n",
       "L 55.582288 64.930753 \n",
       "L 56.257574 65.738134 \n",
       "L 56.932465 66.523076 \n",
       "L 57.606992 67.285616 \n",
       "L 58.281191 68.025793 \n",
       "L 58.955099 68.743644 \n",
       "L 59.628746 69.439201 \n",
       "L 60.302172 70.112498 \n",
       "L 60.975408 70.763564 \n",
       "L 61.648489 71.392423 \n",
       "L 62.32145 71.999103 \n",
       "L 62.994327 72.583625 \n",
       "L 63.667153 73.146009 \n",
       "L 64.339963 73.686274 \n",
       "L 65.012792 74.204433 \n",
       "L 65.685675 74.700503 \n",
       "L 66.358646 75.174493 \n",
       "L 67.03174 75.626411 \n",
       "L 67.704992 76.056265 \n",
       "L 68.378436 76.464059 \n",
       "L 69.052109 76.849795 \n",
       "L 69.726044 77.213472 \n",
       "L 70.400277 77.555088 \n",
       "L 71.074842 77.874638 \n",
       "L 71.749776 78.172114 \n",
       "L 72.425114 78.447509 \n",
       "L 73.10089 78.700809 \n",
       "L 73.77714 78.932 \n",
       "L 74.453901 79.141067 \n",
       "L 75.131207 79.327991 \n",
       "L 75.809095 79.49275 \n",
       "L 76.487601 79.635321 \n",
       "L 77.166761 79.755679 \n",
       "L 77.84661 79.853796 \n",
       "L 78.527187 79.929641 \n",
       "L 79.208526 79.983181 \n",
       "L 79.890665 80.014381 \n",
       "L 80.573641 80.023203 \n",
       "L 81.25749 80.009608 \n",
       "L 81.94225 79.973553 \n",
       "L 82.627958 79.914993 \n",
       "L 83.314653 79.833881 \n",
       "L 84.002372 79.730167 \n",
       "L 84.691153 79.603798 \n",
       "L 85.381033 79.454721 \n",
       "L 86.072053 79.282876 \n",
       "L 86.764249 79.088205 \n",
       "L 87.457663 78.870644 \n",
       "L 88.152332 78.630129 \n",
       "L 88.848296 78.366592 \n",
       "L 89.545594 78.079963 \n",
       "L 90.244268 77.770169 \n",
       "L 90.944357 77.437133 \n",
       "L 91.645902 77.080777 \n",
       "L 92.348943 76.701021 \n",
       "L 93.053522 76.297781 \n",
       "L 93.759682 75.870968 \n",
       "L 94.467461 75.420496 \n",
       "L 95.176906 74.946268 \n",
       "L 95.888054 74.448193 \n",
       "L 96.600953 73.926169 \n",
       "L 97.315642 73.380101 \n",
       "L 98.032166 72.809879 \n",
       "L 98.750572 72.215397 \n",
       "L 99.470899 71.596549 \n",
       "L 100.193195 70.953216 \n",
       "L 100.917504 70.285286 \n",
       "L 101.643872 69.592639 \n",
       "L 102.372347 68.875146 \n",
       "L 103.102972 68.13269 \n",
       "L 103.835796 67.365139 \n",
       "L 104.570867 66.572357 \n",
       "L 105.308229 65.754212 \n",
       "L 106.047934 64.910565 \n",
       "L 106.790031 64.041268 \n",
       "L 107.534566 63.146182 \n",
       "L 108.281591 62.225153 \n",
       "L 109.031158 61.278027 \n",
       "L 109.783314 60.30465 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 44.704077 37.130816 \n",
       "L 45.39059 38.439577 \n",
       "L 46.076006 39.724499 \n",
       "L 46.760358 40.985669 \n",
       "L 47.443683 42.223179 \n",
       "L 48.126019 43.437122 \n",
       "L 48.8074 44.627576 \n",
       "L 49.487862 45.794628 \n",
       "L 50.167441 46.938357 \n",
       "L 50.846173 48.058845 \n",
       "L 51.524095 49.156164 \n",
       "L 52.201239 50.230388 \n",
       "L 52.877641 51.281586 \n",
       "L 53.55334 52.30983 \n",
       "L 54.228364 53.315182 \n",
       "L 54.902753 54.297706 \n",
       "L 55.576542 55.257467 \n",
       "L 56.249762 56.194518 \n",
       "L 56.922452 57.10892 \n",
       "L 57.594642 58.000724 \n",
       "L 58.266369 58.869982 \n",
       "L 58.937667 59.716743 \n",
       "L 59.608573 60.541059 \n",
       "L 60.279117 61.342969 \n",
       "L 60.949338 62.122518 \n",
       "L 61.619266 62.879745 \n",
       "L 62.288935 63.614687 \n",
       "L 62.958383 64.327383 \n",
       "L 63.627642 65.017862 \n",
       "L 64.296746 65.686159 \n",
       "L 64.965731 66.3323 \n",
       "L 65.634628 66.956313 \n",
       "L 66.303473 67.558222 \n",
       "L 66.972301 68.138049 \n",
       "L 67.641145 68.695813 \n",
       "L 68.31004 69.231533 \n",
       "L 68.97902 69.745224 \n",
       "L 69.64812 70.236898 \n",
       "L 70.317373 70.706567 \n",
       "L 70.986814 71.154239 \n",
       "L 71.656477 71.57992 \n",
       "L 72.326397 71.983615 \n",
       "L 72.996609 72.365325 \n",
       "L 73.667147 72.72505 \n",
       "L 74.338046 73.062786 \n",
       "L 75.00934 73.378531 \n",
       "L 75.681065 73.672275 \n",
       "L 76.353256 73.944009 \n",
       "L 77.025947 74.193723 \n",
       "L 77.699174 74.421401 \n",
       "L 78.372972 74.627027 \n",
       "L 79.047376 74.810583 \n",
       "L 79.722423 74.972048 \n",
       "L 80.398147 75.111399 \n",
       "L 81.074586 75.22861 \n",
       "L 81.751773 75.323652 \n",
       "L 82.429748 75.396497 \n",
       "L 83.108543 75.44711 \n",
       "L 83.788197 75.475458 \n",
       "L 84.468746 75.481503 \n",
       "L 85.150226 75.465205 \n",
       "L 85.832676 75.426522 \n",
       "L 86.516131 75.365409 \n",
       "L 87.20063 75.28182 \n",
       "L 87.88621 75.175704 \n",
       "L 88.572909 75.047011 \n",
       "L 89.260763 74.895685 \n",
       "L 89.949814 74.721669 \n",
       "L 90.640097 74.524905 \n",
       "L 91.331653 74.30533 \n",
       "L 92.02452 74.062879 \n",
       "L 92.718737 73.797485 \n",
       "L 93.414344 73.50908 \n",
       "L 94.111381 73.197589 \n",
       "L 94.809887 72.862938 \n",
       "L 95.509904 72.505049 \n",
       "L 96.211471 72.123843 \n",
       "L 96.91463 71.719235 \n",
       "L 97.619424 71.291139 \n",
       "L 98.32589 70.839468 \n",
       "L 99.034075 70.364128 \n",
       "L 99.744017 69.865027 \n",
       "L 100.455762 69.342066 \n",
       "L 101.16935 68.795147 \n",
       "L 101.884825 68.224165 \n",
       "L 102.602234 67.629013 \n",
       "L 103.321617 67.009586 \n",
       "L 104.04302 66.365767 \n",
       "L 104.766488 65.697445 \n",
       "L 105.492066 65.0045 \n",
       "L 106.219803 64.286806 \n",
       "L 106.949741 63.544246 \n",
       "L 107.681928 62.776689 \n",
       "L 108.416413 61.984001 \n",
       "L 109.15324 61.166051 \n",
       "L 109.892459 60.322702 \n",
       "L 110.634122 59.453807 \n",
       "L 111.378272 58.559227 \n",
       "L 112.124961 57.638812 \n",
       "L 112.874242 56.692409 \n",
       "L 113.626162 55.719865 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 48.846937 35.215521 \n",
       "L 49.525936 36.513831 \n",
       "L 50.203924 37.788482 \n",
       "L 50.880931 39.039559 \n",
       "L 51.556996 40.267153 \n",
       "L 52.232156 41.471354 \n",
       "L 52.906442 42.65224 \n",
       "L 53.579891 43.809896 \n",
       "L 54.252538 44.9444 \n",
       "L 54.924418 46.055833 \n",
       "L 55.595568 47.144266 \n",
       "L 56.266017 48.20977 \n",
       "L 56.935804 49.252415 \n",
       "L 57.604963 50.27227 \n",
       "L 58.273526 51.269398 \n",
       "L 58.941528 52.24386 \n",
       "L 59.609006 53.195721 \n",
       "L 60.27599 54.125034 \n",
       "L 60.942517 55.031857 \n",
       "L 61.608619 55.916241 \n",
       "L 62.27433 56.778238 \n",
       "L 62.939684 57.617894 \n",
       "L 63.604717 58.435263 \n",
       "L 64.26946 59.23038 \n",
       "L 64.933949 60.003293 \n",
       "L 65.598216 60.754037 \n",
       "L 66.262294 61.482651 \n",
       "L 66.926218 62.189171 \n",
       "L 67.590021 62.873628 \n",
       "L 68.253737 63.536054 \n",
       "L 68.917401 64.176477 \n",
       "L 69.581043 64.794921 \n",
       "L 70.244699 65.391412 \n",
       "L 70.908404 65.965972 \n",
       "L 71.57219 66.51862 \n",
       "L 72.236091 67.049372 \n",
       "L 72.90014 67.558244 \n",
       "L 73.564372 68.045249 \n",
       "L 74.228821 68.510397 \n",
       "L 74.89352 68.953697 \n",
       "L 75.558503 69.375154 \n",
       "L 76.223804 69.774773 \n",
       "L 76.889458 70.152556 \n",
       "L 77.555498 70.508501 \n",
       "L 78.221959 70.842607 \n",
       "L 78.888875 71.154868 \n",
       "L 79.556281 71.445277 \n",
       "L 80.224211 71.713824 \n",
       "L 80.892699 71.960499 \n",
       "L 81.561781 72.185287 \n",
       "L 82.231491 72.388172 \n",
       "L 82.901864 72.569135 \n",
       "L 83.572936 72.728156 \n",
       "L 84.244741 72.865212 \n",
       "L 84.917316 72.980278 \n",
       "L 85.590695 73.073326 \n",
       "L 86.264916 73.144327 \n",
       "L 86.940012 73.193247 \n",
       "L 87.61602 73.220053 \n",
       "L 88.292976 73.224708 \n",
       "L 88.970917 73.207172 \n",
       "L 89.64988 73.167404 \n",
       "L 90.3299 73.10536 \n",
       "L 91.011016 73.020993 \n",
       "L 91.693265 72.914256 \n",
       "L 92.376683 72.785096 \n",
       "L 93.061308 72.63346 \n",
       "L 93.747178 72.459292 \n",
       "L 94.434331 72.262533 \n",
       "L 95.122807 72.043122 \n",
       "L 95.812643 71.800995 \n",
       "L 96.503877 71.536087 \n",
       "L 97.196549 71.248328 \n",
       "L 97.8907 70.937647 \n",
       "L 98.586367 70.603971 \n",
       "L 99.283593 70.247222 \n",
       "L 99.982415 69.867321 \n",
       "L 100.682875 69.464187 \n",
       "L 101.385016 69.037734 \n",
       "L 102.088876 68.587877 \n",
       "L 102.794499 68.114521 \n",
       "L 103.501924 67.617578 \n",
       "L 104.211197 67.096949 \n",
       "L 104.922357 66.552539 \n",
       "L 105.635448 65.984243 \n",
       "L 106.350516 65.391957 \n",
       "L 107.067601 64.775577 \n",
       "L 107.786749 64.134987 \n",
       "L 108.508005 63.470079 \n",
       "L 109.231412 62.780734 \n",
       "L 109.95702 62.066829 \n",
       "L 110.68487 61.328248 \n",
       "L 111.415009 60.564862 \n",
       "L 112.147488 59.77654 \n",
       "L 112.882348 58.963152 \n",
       "L 113.61964 58.124563 \n",
       "L 114.359415 57.26063 \n",
       "L 115.101716 56.371215 \n",
       "L 115.846594 55.456171 \n",
       "L 116.594103 54.515346 \n",
       "L 117.344288 53.548592 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 52.962197 35.582887 \n",
       "L 53.632693 36.867868 \n",
       "L 54.302263 38.129446 \n",
       "L 54.970938 39.367705 \n",
       "L 55.638754 40.582734 \n",
       "L 56.305748 41.774621 \n",
       "L 56.971951 42.943441 \n",
       "L 57.637398 44.08928 \n",
       "L 58.302124 45.212212 \n",
       "L 58.966162 46.312317 \n",
       "L 59.629549 47.389666 \n",
       "L 60.292314 48.444326 \n",
       "L 60.954494 49.476368 \n",
       "L 61.616123 50.485859 \n",
       "L 62.277231 51.47286 \n",
       "L 62.937854 52.43743 \n",
       "L 63.598027 53.379635 \n",
       "L 64.25778 54.299524 \n",
       "L 64.917149 55.197158 \n",
       "L 65.576164 56.072583 \n",
       "L 66.23486 56.925851 \n",
       "L 66.89327 57.757009 \n",
       "L 67.55143 58.566107 \n",
       "L 68.209368 59.353182 \n",
       "L 68.867121 60.118278 \n",
       "L 69.52472 60.861433 \n",
       "L 70.182197 61.582682 \n",
       "L 70.839587 62.282062 \n",
       "L 71.496921 62.959602 \n",
       "L 72.154235 63.615335 \n",
       "L 72.81156 64.249288 \n",
       "L 73.468929 64.861484 \n",
       "L 74.126375 65.451949 \n",
       "L 74.783932 66.020704 \n",
       "L 75.441631 66.567767 \n",
       "L 76.099508 67.093156 \n",
       "L 76.757594 67.596885 \n",
       "L 77.415923 68.078966 \n",
       "L 78.074529 68.539411 \n",
       "L 78.733443 68.978226 \n",
       "L 79.392701 69.395419 \n",
       "L 80.052334 69.790993 \n",
       "L 80.712378 70.164949 \n",
       "L 81.372865 70.517289 \n",
       "L 82.033828 70.848007 \n",
       "L 82.695303 71.1571 \n",
       "L 83.357322 71.444561 \n",
       "L 84.019919 71.710381 \n",
       "L 84.68313 71.954548 \n",
       "L 85.346987 72.177048 \n",
       "L 86.011525 72.377866 \n",
       "L 86.676779 72.556983 \n",
       "L 87.342783 72.71438 \n",
       "L 88.009572 72.850033 \n",
       "L 88.677181 72.96392 \n",
       "L 89.345644 73.05601 \n",
       "L 90.014999 73.126277 \n",
       "L 90.685277 73.174688 \n",
       "L 91.356517 73.20121 \n",
       "L 92.028753 73.205806 \n",
       "L 92.702021 73.188438 \n",
       "L 93.376357 73.149065 \n",
       "L 94.051798 73.087645 \n",
       "L 94.72838 73.004131 \n",
       "L 95.40614 72.898476 \n",
       "L 96.085115 72.77063 \n",
       "L 96.76534 72.62054 \n",
       "L 97.446856 72.448152 \n",
       "L 98.129697 72.253407 \n",
       "L 98.813903 72.036247 \n",
       "L 99.499512 71.796607 \n",
       "L 100.186561 71.534425 \n",
       "L 100.87509 71.249633 \n",
       "L 101.565137 70.94216 \n",
       "L 102.256741 70.611934 \n",
       "L 102.949944 70.25888 \n",
       "L 103.644781 69.882921 \n",
       "L 104.341296 69.483977 \n",
       "L 105.03953 69.061962 \n",
       "L 105.73952 68.616795 \n",
       "L 106.44131 68.148384 \n",
       "L 107.144939 67.65664 \n",
       "L 107.850452 67.141469 \n",
       "L 108.557887 66.602774 \n",
       "L 109.267289 66.040457 \n",
       "L 109.978702 65.454413 \n",
       "L 110.692165 64.84454 \n",
       "L 111.407725 64.210727 \n",
       "L 112.125425 63.552866 \n",
       "L 112.84531 62.870843 \n",
       "L 113.567427 62.164534 \n",
       "L 114.291816 61.433828 \n",
       "L 115.018526 60.678598 \n",
       "L 115.747605 59.898717 \n",
       "L 116.479095 59.094056 \n",
       "L 117.213046 58.264485 \n",
       "L 117.949508 57.409862 \n",
       "L 118.688523 56.530053 \n",
       "L 119.430144 55.624914 \n",
       "L 120.174422 54.694297 \n",
       "L 120.921401 53.738056 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 57.030186 38.162323 \n",
       "L 57.691268 39.431278 \n",
       "L 58.35151 40.677158 \n",
       "L 59.010941 41.900045 \n",
       "L 59.669597 43.100025 \n",
       "L 60.327513 44.277186 \n",
       "L 60.984719 45.4316 \n",
       "L 61.64125 46.563351 \n",
       "L 62.297139 47.672511 \n",
       "L 62.95242 48.759159 \n",
       "L 63.607128 49.823362 \n",
       "L 64.26129 50.865188 \n",
       "L 64.914943 51.884705 \n",
       "L 65.568121 52.881979 \n",
       "L 66.220853 53.857068 \n",
       "L 66.873173 54.810032 \n",
       "L 67.525116 55.740933 \n",
       "L 68.17671 56.649821 \n",
       "L 68.827992 57.536754 \n",
       "L 69.47899 58.401777 \n",
       "L 70.129739 59.244942 \n",
       "L 70.78027 60.066294 \n",
       "L 71.430619 60.865882 \n",
       "L 72.080813 61.643742 \n",
       "L 72.730889 62.399918 \n",
       "L 73.380876 63.134445 \n",
       "L 74.030806 63.847359 \n",
       "L 74.680713 64.538695 \n",
       "L 75.330629 65.208482 \n",
       "L 75.980586 65.856753 \n",
       "L 76.630616 66.483532 \n",
       "L 77.28075 67.088844 \n",
       "L 77.931023 67.672714 \n",
       "L 78.581466 68.235161 \n",
       "L 79.232111 68.776204 \n",
       "L 79.882991 69.29586 \n",
       "L 80.534138 69.794142 \n",
       "L 81.185586 70.271064 \n",
       "L 81.837365 70.726636 \n",
       "L 82.48951 71.160864 \n",
       "L 83.142052 71.573757 \n",
       "L 83.795024 71.965316 \n",
       "L 84.448459 72.335545 \n",
       "L 85.102391 72.684442 \n",
       "L 85.756851 73.012006 \n",
       "L 86.411874 73.318231 \n",
       "L 87.067492 73.60311 \n",
       "L 87.723739 73.866634 \n",
       "L 88.380648 74.108794 \n",
       "L 89.038253 74.329574 \n",
       "L 89.696587 74.528961 \n",
       "L 90.355684 74.706936 \n",
       "L 91.015578 74.863481 \n",
       "L 91.676303 74.998571 \n",
       "L 92.337894 75.112186 \n",
       "L 93.000384 75.204296 \n",
       "L 93.663809 75.274874 \n",
       "L 94.328202 75.32389 \n",
       "L 94.9936 75.35131 \n",
       "L 95.660035 75.3571 \n",
       "L 96.327545 75.341221 \n",
       "L 96.996163 75.303635 \n",
       "L 97.665927 75.2443 \n",
       "L 98.336873 75.16317 \n",
       "L 99.009035 75.0602 \n",
       "L 99.68245 74.935341 \n",
       "L 100.357154 74.788542 \n",
       "L 101.033185 74.619749 \n",
       "L 101.710578 74.428906 \n",
       "L 102.389372 74.215955 \n",
       "L 103.069605 73.980835 \n",
       "L 103.751312 73.723484 \n",
       "L 104.434532 73.443835 \n",
       "L 105.119305 73.14182 \n",
       "L 105.805668 72.81737 \n",
       "L 106.49366 72.47041 \n",
       "L 107.183319 72.100867 \n",
       "L 107.874686 71.70866 \n",
       "L 108.567803 71.29371 \n",
       "L 109.262704 70.855934 \n",
       "L 109.959435 70.395243 \n",
       "L 110.658033 69.911553 \n",
       "L 111.358542 69.404769 \n",
       "L 112.061 68.8748 \n",
       "L 112.76545 68.321548 \n",
       "L 113.471937 67.744912 \n",
       "L 114.1805 67.144793 \n",
       "L 114.891182 66.521082 \n",
       "L 115.604029 65.873674 \n",
       "L 116.319082 65.202458 \n",
       "L 117.036389 64.507314 \n",
       "L 117.75599 63.788135 \n",
       "L 118.477932 63.044796 \n",
       "L 119.202262 62.277172 \n",
       "L 119.929023 61.485141 \n",
       "L 120.658262 60.668573 \n",
       "L 121.39003 59.827333 \n",
       "L 122.124368 58.961289 \n",
       "L 122.861327 58.0703 \n",
       "L 123.600958 57.154224 \n",
       "L 124.343305 56.212918 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 61.032539 42.864518 \n",
       "L 61.683382 44.114955 \n",
       "L 62.333469 45.342712 \n",
       "L 62.982828 46.547867 \n",
       "L 63.631494 47.730506 \n",
       "L 64.279502 48.890712 \n",
       "L 64.926879 50.028557 \n",
       "L 65.573661 51.144122 \n",
       "L 66.219879 52.237477 \n",
       "L 66.865565 53.308698 \n",
       "L 67.510754 54.357851 \n",
       "L 68.155474 55.385003 \n",
       "L 68.799758 56.390218 \n",
       "L 69.443641 57.373561 \n",
       "L 70.087149 58.33509 \n",
       "L 70.730317 59.274861 \n",
       "L 71.373179 60.192936 \n",
       "L 72.015762 61.089362 \n",
       "L 72.658101 61.964197 \n",
       "L 73.300224 62.817484 \n",
       "L 73.942165 63.649275 \n",
       "L 74.583955 64.459611 \n",
       "L 75.225627 65.248542 \n",
       "L 75.867209 66.016102 \n",
       "L 76.508736 66.762334 \n",
       "L 77.150238 67.487273 \n",
       "L 77.791744 68.190953 \n",
       "L 78.433288 68.873408 \n",
       "L 79.074901 69.534668 \n",
       "L 79.716615 70.174762 \n",
       "L 80.35846 70.793717 \n",
       "L 81.000468 71.391555 \n",
       "L 81.642671 71.968301 \n",
       "L 82.2851 72.523973 \n",
       "L 82.927787 73.05859 \n",
       "L 83.570764 73.572171 \n",
       "L 84.214061 74.064724 \n",
       "L 84.857712 74.536266 \n",
       "L 85.501747 74.986806 \n",
       "L 86.146198 75.416352 \n",
       "L 86.791098 75.824908 \n",
       "L 87.436478 76.21248 \n",
       "L 88.08237 76.579068 \n",
       "L 88.728807 76.924675 \n",
       "L 89.375821 77.249294 \n",
       "L 90.023443 77.552924 \n",
       "L 90.671707 77.835557 \n",
       "L 91.320646 78.097185 \n",
       "L 91.970291 78.337796 \n",
       "L 92.620676 78.55738 \n",
       "L 93.271833 78.755921 \n",
       "L 93.923795 78.933401 \n",
       "L 94.576597 79.089802 \n",
       "L 95.23027 79.225104 \n",
       "L 95.884849 79.339282 \n",
       "L 96.540367 79.43231 \n",
       "L 97.196859 79.504163 \n",
       "L 97.854357 79.55481 \n",
       "L 98.512896 79.584217 \n",
       "L 99.17251 79.592354 \n",
       "L 99.833233 79.579182 \n",
       "L 100.495101 79.544666 \n",
       "L 101.158148 79.48876 \n",
       "L 101.82241 79.411427 \n",
       "L 102.487922 79.312618 \n",
       "L 103.154719 79.192287 \n",
       "L 103.822835 79.050385 \n",
       "L 104.49231 78.886859 \n",
       "L 105.163176 78.701658 \n",
       "L 105.835473 78.494721 \n",
       "L 106.509236 78.265991 \n",
       "L 107.184501 78.015409 \n",
       "L 107.861306 77.742909 \n",
       "L 108.539689 77.448426 \n",
       "L 109.219687 77.131892 \n",
       "L 109.90134 76.793237 \n",
       "L 110.584683 76.432386 \n",
       "L 111.269757 76.049265 \n",
       "L 111.956602 75.643794 \n",
       "L 112.645253 75.215895 \n",
       "L 113.335755 74.765482 \n",
       "L 114.028142 74.292471 \n",
       "L 114.72246 73.796773 \n",
       "L 115.418745 73.278298 \n",
       "L 116.117039 72.736952 \n",
       "L 116.817387 72.172637 \n",
       "L 117.519825 71.585258 \n",
       "L 118.224399 70.974708 \n",
       "L 118.93115 70.340886 \n",
       "L 119.64012 69.683686 \n",
       "L 120.351357 69.00299 \n",
       "L 121.064899 68.298694 \n",
       "L 121.780792 67.570679 \n",
       "L 122.499083 66.818824 \n",
       "L 123.219813 66.043009 \n",
       "L 123.943029 65.243109 \n",
       "L 124.66878 64.418994 \n",
       "L 125.397108 63.570536 \n",
       "L 126.128062 62.697599 \n",
       "L 126.861691 61.800045 \n",
       "L 127.598039 60.877736 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "    <path d=\"M 64.952423 49.583939 \n",
       "L 65.59229 50.813593 \n",
       "L 66.231483 52.02102 \n",
       "L 66.870029 53.206298 \n",
       "L 67.507962 54.369509 \n",
       "L 68.145316 55.510735 \n",
       "L 68.782117 56.630043 \n",
       "L 69.4184 57.727512 \n",
       "L 70.054194 58.803211 \n",
       "L 70.689533 59.857213 \n",
       "L 71.324447 60.889583 \n",
       "L 71.958965 61.900383 \n",
       "L 72.593119 62.889679 \n",
       "L 73.226942 63.857531 \n",
       "L 73.86046 64.803996 \n",
       "L 74.493708 65.729129 \n",
       "L 75.126716 66.632989 \n",
       "L 75.759513 67.515623 \n",
       "L 76.392131 68.377085 \n",
       "L 77.024599 69.21742 \n",
       "L 77.656949 70.036674 \n",
       "L 78.28921 70.834892 \n",
       "L 78.921416 71.612118 \n",
       "L 79.553594 72.368387 \n",
       "L 80.185776 73.103741 \n",
       "L 80.817992 73.818212 \n",
       "L 81.450272 74.511837 \n",
       "L 82.082648 75.184646 \n",
       "L 82.715148 75.836667 \n",
       "L 83.347805 76.467931 \n",
       "L 83.980649 77.078464 \n",
       "L 84.61371 77.668285 \n",
       "L 85.247018 78.237421 \n",
       "L 85.880606 78.785889 \n",
       "L 86.514502 79.313707 \n",
       "L 87.148739 79.820891 \n",
       "L 87.783346 80.307455 \n",
       "L 88.418355 80.773411 \n",
       "L 89.053798 81.218769 \n",
       "L 89.689703 81.643536 \n",
       "L 90.326103 82.047717 \n",
       "L 90.963029 82.431319 \n",
       "L 91.600512 82.794341 \n",
       "L 92.238583 83.136783 \n",
       "L 92.877274 83.458645 \n",
       "L 93.516616 83.759922 \n",
       "L 94.156642 84.040607 \n",
       "L 94.797382 84.300692 \n",
       "L 95.438868 84.540167 \n",
       "L 96.081133 84.759022 \n",
       "L 96.724208 84.95724 \n",
       "L 97.368126 85.134806 \n",
       "L 98.012919 85.291702 \n",
       "L 98.658619 85.427908 \n",
       "L 99.30526 85.543401 \n",
       "L 99.952874 85.638157 \n",
       "L 100.601495 85.712149 \n",
       "L 101.251153 85.765349 \n",
       "L 101.901885 85.797728 \n",
       "L 102.553721 85.809251 \n",
       "L 103.206697 85.799883 \n",
       "L 103.860847 85.76959 \n",
       "L 104.516204 85.71833 \n",
       "L 105.172803 85.646064 \n",
       "L 105.830677 85.552747 \n",
       "L 106.489863 85.438334 \n",
       "L 107.150393 85.302779 \n",
       "L 107.812304 85.146031 \n",
       "L 108.475631 84.968036 \n",
       "L 109.14041 84.768742 \n",
       "L 109.806676 84.548092 \n",
       "L 110.474465 84.306027 \n",
       "L 111.143813 84.042487 \n",
       "L 111.814759 83.757406 \n",
       "L 112.487337 83.450722 \n",
       "L 113.161586 83.122362 \n",
       "L 113.837542 82.77226 \n",
       "L 114.515243 82.400341 \n",
       "L 115.19473 82.00653 \n",
       "L 115.876036 81.590751 \n",
       "L 116.559205 81.152921 \n",
       "L 117.244272 80.69296 \n",
       "L 117.931279 80.210782 \n",
       "L 118.620262 79.706298 \n",
       "L 119.311264 79.179421 \n",
       "L 120.004327 78.630055 \n",
       "L 120.699487 78.058108 \n",
       "L 121.396788 77.463478 \n",
       "L 122.09627 76.846068 \n",
       "L 122.797977 76.205775 \n",
       "L 123.501952 75.542486 \n",
       "L 124.208234 74.8561 \n",
       "L 124.916868 74.146503 \n",
       "L 125.6279 73.413579 \n",
       "L 126.341369 72.657212 \n",
       "L 127.057322 71.877282 \n",
       "L 127.775806 71.073663 \n",
       "L 128.496862 70.246234 \n",
       "L 129.220538 69.394861 \n",
       "L 129.946883 68.519414 \n",
       "L 130.675939 67.619758 \n",
       "\" clip-path=\"url(#p5979f259ce)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p5979f259ce\">\n",
       "   <rect x=\"7.2\" y=\"7.2\" width=\"138.6\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x, y = torch.meshgrid(\n",
    "    torch.linspace(-1.0, 1.0, 101), torch.linspace(-1.0, 1.0, 101))\n",
    "z = x**2 - y**2\n",
    "\n",
    "ax = d2l.plt.figure().add_subplot(111, projection='3d')\n",
    "ax.plot_wireframe(x, y, z, **{'rstride': 10, 'cstride': 10})\n",
    "ax.plot([0], [0], [0], 'rx')\n",
    "ticks = [-1, 0, 1]\n",
    "d2l.plt.xticks(ticks)\n",
    "d2l.plt.yticks(ticks)\n",
    "ax.set_zticks(ticks)\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('y');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f52a48d2",
   "metadata": {
    "origin_pos": 19
   },
   "source": [
    "我们假设函数的输入是$k$维向量，其输出是标量，因此其Hessian矩阵（也称黑塞矩阵）将有$k$个特征值（参考[特征分解的在线附录](https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/eigendecomposition.html))。函数的解可能是局部最小值、局部最大值或函数梯度为零位置处的鞍点：\n",
    "\n",
    "* 当函数在零梯度位置处的Hessian矩阵的特征值全部为正值时，我们有该函数的局部最小值；\n",
    "* 当函数在零梯度位置处的Hessian矩阵的特征值全部为负值时，我们有该函数的局部最大值；\n",
    "* 当函数在零梯度位置处的Hessian矩阵的特征值为负值和正值时，我们有该函数的一个鞍点。\n",
    "\n",
    "对于高维度问题，至少*部分*特征值为负的可能性相当高。这使得鞍点比局部最小值更有可能出现。我们将在下一节介绍凸性时讨论这种情况的一些例外。简而言之，凸函数是Hessian函数的特征值永远不为负值的函数。不幸的是，大多数深度学习问题并不属于这一类。尽管如此，它还是研究优化算法的一个很好的工具。\n",
    "\n",
    "### 梯度消失\n",
    "\n",
    "可能遇到的最隐蔽问题是梯度消失。回想一下我们在 :numref:`subsec_activation_functions`中常用的激活函数及其衍生函数。例如，假设我们想最小化函数$f(x) = \\tanh(x)$，然后我们恰好从$x = 4$开始。正如我们所看到的那样，$f$的梯度接近零。更具体地说，$f'(x) = 1 - \\tanh^2(x)$，因此是$f'(4) = 0.0013$。因此，在我们取得进展之前，优化将会停滞很长一段时间。事实证明，这是在引入ReLU激活函数之前训练深度学习模型相当棘手的原因之一。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1682addf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-07T16:35:20.066904Z",
     "iopub.status.busy": "2022-12-07T16:35:20.066350Z",
     "iopub.status.idle": "2022-12-07T16:35:20.343856Z",
     "shell.execute_reply": "2022-12-07T16:35:20.343076Z"
    },
    "origin_pos": 20,
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"263.314464pt\" height=\"183.35625pt\" viewBox=\"0 0 263.314464 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2022-12-07T16:35:20.304044</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.5.1, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 263.314464 183.35625 \n",
       "L 263.314464 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 52.160938 145.8 \n",
       "L 247.460938 145.8 \n",
       "L 247.460938 7.2 \n",
       "L 52.160938 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 61.03821 145.8 \n",
       "L 61.03821 7.2 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m1d906887cd\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1d906887cd\" x=\"61.03821\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −2 -->\n",
       "      <g transform=\"translate(53.667116 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \n",
       "L 4684 2272 \n",
       "L 4684 1741 \n",
       "L 678 1741 \n",
       "L 678 2272 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-32\" x=\"83.789062\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 111.838056 145.8 \n",
       "L 111.838056 7.2 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1d906887cd\" x=\"111.838056\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(108.656806 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 162.637901 145.8 \n",
       "L 162.637901 7.2 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1d906887cd\" x=\"162.637901\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(159.456651 160.398438)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 213.437747 145.8 \n",
       "L 213.437747 7.2 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1d906887cd\" x=\"213.437747\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(210.256497 160.398438)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_5\">\n",
       "     <!-- x -->\n",
       "     <g transform=\"translate(146.851563 174.076563)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-78\" d=\"M 3513 3500 \n",
       "L 2247 1797 \n",
       "L 3578 0 \n",
       "L 2900 0 \n",
       "L 1881 1375 \n",
       "L 863 0 \n",
       "L 184 0 \n",
       "L 1544 1831 \n",
       "L 300 3500 \n",
       "L 978 3500 \n",
       "L 1906 2253 \n",
       "L 2834 3500 \n",
       "L 3513 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-78\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 52.160938 141.807879 \n",
       "L 247.460938 141.807879 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <defs>\n",
       "       <path id=\"mbd92fac06d\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mbd92fac06d\" x=\"52.160938\" y=\"141.807879\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- −1.0 -->\n",
       "      <g transform=\"translate(20.878125 145.607098)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-31\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"179.199219\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 52.160938 109.729424 \n",
       "L 247.460938 109.729424 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mbd92fac06d\" x=\"52.160938\" y=\"109.729424\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- −0.5 -->\n",
       "      <g transform=\"translate(20.878125 113.528642)scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-2212\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"83.789062\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"147.412109\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"179.199219\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 52.160938 77.650968 \n",
       "L 247.460938 77.650968 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mbd92fac06d\" x=\"52.160938\" y=\"77.650968\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(29.257812 81.450187)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 52.160938 45.572513 \n",
       "L 247.460938 45.572513 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mbd92fac06d\" x=\"52.160938\" y=\"45.572513\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(29.257812 49.371732)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 52.160938 13.494057 \n",
       "L 247.460938 13.494057 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mbd92fac06d\" x=\"52.160938\" y=\"13.494057\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(29.257812 17.293276)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- f(x) -->\n",
       "     <g transform=\"translate(14.798438 85.121094)rotate(-90)scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-66\" d=\"M 2375 4863 \n",
       "L 2375 4384 \n",
       "L 1825 4384 \n",
       "Q 1516 4384 1395 4259 \n",
       "Q 1275 4134 1275 3809 \n",
       "L 1275 3500 \n",
       "L 2222 3500 \n",
       "L 2222 3053 \n",
       "L 1275 3053 \n",
       "L 1275 0 \n",
       "L 697 0 \n",
       "L 697 3053 \n",
       "L 147 3053 \n",
       "L 147 3500 \n",
       "L 697 3500 \n",
       "L 697 3744 \n",
       "Q 697 4328 969 4595 \n",
       "Q 1241 4863 1831 4863 \n",
       "L 2375 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-28\" d=\"M 1984 4856 \n",
       "Q 1566 4138 1362 3434 \n",
       "Q 1159 2731 1159 2009 \n",
       "Q 1159 1288 1364 580 \n",
       "Q 1569 -128 1984 -844 \n",
       "L 1484 -844 \n",
       "Q 1016 -109 783 600 \n",
       "Q 550 1309 550 2009 \n",
       "Q 550 2706 781 3412 \n",
       "Q 1013 4119 1484 4856 \n",
       "L 1984 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-29\" d=\"M 513 4856 \n",
       "L 1013 4856 \n",
       "Q 1481 4119 1714 3412 \n",
       "Q 1947 2706 1947 2009 \n",
       "Q 1947 1309 1714 600 \n",
       "Q 1481 -109 1013 -844 \n",
       "L 513 -844 \n",
       "Q 928 -128 1133 580 \n",
       "Q 1338 1288 1338 2009 \n",
       "Q 1338 2731 1133 3434 \n",
       "Q 928 4138 513 4856 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-66\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-28\" x=\"35.205078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-78\" x=\"74.21875\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-29\" x=\"133.398438\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_19\">\n",
       "    <path d=\"M 61.03821 139.5 \n",
       "L 64.848201 138.712047 \n",
       "L 68.150188 137.821364 \n",
       "L 71.198179 136.782374 \n",
       "L 73.992171 135.605492 \n",
       "L 76.53216 134.312485 \n",
       "L 79.072156 132.769871 \n",
       "L 81.358147 131.135675 \n",
       "L 83.644141 129.23713 \n",
       "L 85.930135 127.043226 \n",
       "L 88.216127 124.523617 \n",
       "L 90.502121 121.65036 \n",
       "L 92.788114 118.400162 \n",
       "L 95.074106 114.756979 \n",
       "L 97.3601 110.714828 \n",
       "L 99.900092 105.764482 \n",
       "L 102.694083 99.798835 \n",
       "L 105.742074 92.759648 \n",
       "L 109.806062 82.7726 \n",
       "L 118.696034 60.737614 \n",
       "L 121.998025 53.274616 \n",
       "L 124.792016 47.500736 \n",
       "L 127.332009 42.741454 \n",
       "L 129.872 38.47183 \n",
       "L 132.157994 35.048422 \n",
       "L 134.443987 32.010145 \n",
       "L 136.729979 29.336587 \n",
       "L 139.015972 27.001577 \n",
       "L 141.301965 24.975587 \n",
       "L 143.587959 23.22771 \n",
       "L 146.127952 21.574409 \n",
       "L 148.667945 20.18608 \n",
       "L 151.461934 18.920419 \n",
       "L 154.509925 17.801452 \n",
       "L 157.811915 16.841004 \n",
       "L 161.367903 16.039842 \n",
       "L 165.43189 15.352775 \n",
       "L 170.257877 14.771011 \n",
       "L 176.09986 14.303146 \n",
       "L 183.465836 13.948324 \n",
       "L 193.625808 13.698571 \n",
       "L 209.881755 13.550986 \n",
       "L 238.583665 13.5 \n",
       "L 238.583665 13.5 \n",
       "\" clip-path=\"url(#p87fdf8c1d6)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 52.160938 145.8 \n",
       "L 52.160938 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 247.460938 145.8 \n",
       "L 247.460938 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 52.160937 145.8 \n",
       "L 247.460938 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 52.160937 7.2 \n",
       "L 247.460938 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 209.960548 66.057911 \n",
       "Q 211.633037 40.775352 213.231728 16.608389 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "    <path d=\"M 210.972059 20.46765 \n",
       "L 213.231728 16.608389 \n",
       "L 214.963336 20.73168 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-linecap: round\"/>\n",
       "   </g>\n",
       "   <g id=\"text_12\">\n",
       "    <!-- vanishing gradient -->\n",
       "    <g transform=\"translate(162.637901 77.650968)scale(0.1 -0.1)\">\n",
       "     <defs>\n",
       "      <path id=\"DejaVuSans-76\" d=\"M 191 3500 \n",
       "L 800 3500 \n",
       "L 1894 563 \n",
       "L 2988 3500 \n",
       "L 3597 3500 \n",
       "L 2284 0 \n",
       "L 1503 0 \n",
       "L 191 3500 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-67\" d=\"M 2906 1791 \n",
       "Q 2906 2416 2648 2759 \n",
       "Q 2391 3103 1925 3103 \n",
       "Q 1463 3103 1205 2759 \n",
       "Q 947 2416 947 1791 \n",
       "Q 947 1169 1205 825 \n",
       "Q 1463 481 1925 481 \n",
       "Q 2391 481 2648 825 \n",
       "Q 2906 1169 2906 1791 \n",
       "z\n",
       "M 3481 434 \n",
       "Q 3481 -459 3084 -895 \n",
       "Q 2688 -1331 1869 -1331 \n",
       "Q 1566 -1331 1297 -1286 \n",
       "Q 1028 -1241 775 -1147 \n",
       "L 775 -588 \n",
       "Q 1028 -725 1275 -790 \n",
       "Q 1522 -856 1778 -856 \n",
       "Q 2344 -856 2625 -561 \n",
       "Q 2906 -266 2906 331 \n",
       "L 2906 616 \n",
       "Q 2728 306 2450 153 \n",
       "Q 2172 0 1784 0 \n",
       "Q 1141 0 747 490 \n",
       "Q 353 981 353 1791 \n",
       "Q 353 2603 747 3093 \n",
       "Q 1141 3584 1784 3584 \n",
       "Q 2172 3584 2450 3431 \n",
       "Q 2728 3278 2906 2969 \n",
       "L 2906 3500 \n",
       "L 3481 3500 \n",
       "L 3481 434 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \n",
       "L 2906 4863 \n",
       "L 3481 4863 \n",
       "L 3481 0 \n",
       "L 2906 0 \n",
       "L 2906 525 \n",
       "Q 2725 213 2448 61 \n",
       "Q 2172 -91 1784 -91 \n",
       "Q 1150 -91 751 415 \n",
       "Q 353 922 353 1747 \n",
       "Q 353 2572 751 3078 \n",
       "Q 1150 3584 1784 3584 \n",
       "Q 2172 3584 2448 3432 \n",
       "Q 2725 3281 2906 2969 \n",
       "z\n",
       "M 947 1747 \n",
       "Q 947 1113 1208 752 \n",
       "Q 1469 391 1925 391 \n",
       "Q 2381 391 2643 752 \n",
       "Q 2906 1113 2906 1747 \n",
       "Q 2906 2381 2643 2742 \n",
       "Q 2381 3103 1925 3103 \n",
       "Q 1469 3103 1208 2742 \n",
       "Q 947 2381 947 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "     </defs>\n",
       "     <use xlink:href=\"#DejaVuSans-76\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-61\" x=\"59.179688\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"120.458984\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"183.837891\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-73\" x=\"211.621094\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-68\" x=\"263.720703\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"327.099609\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"354.882812\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-67\" x=\"418.261719\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-20\" x=\"481.738281\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-67\" x=\"513.525391\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-72\" x=\"577.001953\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-61\" x=\"618.115234\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-64\" x=\"679.394531\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-69\" x=\"742.871094\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-65\" x=\"770.654297\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-6e\" x=\"832.177734\"/>\n",
       "     <use xlink:href=\"#DejaVuSans-74\" x=\"895.556641\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p87fdf8c1d6\">\n",
       "   <rect x=\"52.160938\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.arange(-2.0, 5.0, 0.01)\n",
    "d2l.plot(x, [torch.tanh(x)], 'x', 'f(x)')\n",
    "annotate('vanishing gradient', (4, 1), (2, 0.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1726ccb3",
   "metadata": {
    "origin_pos": 22
   },
   "source": [
    "正如我们所看到的那样，深度学习的优化充满挑战。幸运的是，有一系列强大的算法表现良好，即使对于初学者也很容易使用。此外，没有必要找到最优解。局部最优解或其近似解仍然非常有用。\n",
    "\n",
    "## 小结\n",
    "\n",
    "* 最小化训练误差并*不能*保证我们找到最佳的参数集来最小化泛化误差。\n",
    "* 优化问题可能有许多局部最小值。\n",
    "* 一个问题可能有很多的鞍点，因为问题通常不是凸的。\n",
    "* 梯度消失可能会导致优化停滞，重参数化通常会有所帮助。对参数进行良好的初始化也可能是有益的。\n",
    "\n",
    "## 练习\n",
    "\n",
    "1. 考虑一个简单的MLP，它有一个隐藏层，比如，隐藏层中维度为$d$和一个输出。证明对于任何局部最小值，至少有$d！$个等效方案。\n",
    "1. 假设我们有一个对称随机矩阵$\\mathbf{M}$，其中条目$M_{ij} = M_{ji}$各自从某种概率分布$p_{ij}$中抽取。此外，假设$p_{ij}(x) = p_{ij}(-x)$，即分布是对称的（详情请参见 :cite:`Wigner.1958`）。\n",
    "    1. 证明特征值的分布也是对称的。也就是说，对于任何特征向量$\\mathbf{v}$，关联的特征值$\\lambda$满足$P(\\lambda > 0) = P(\\lambda < 0)$的概率为$P(\\lambda > 0) = P(\\lambda < 0)$。\n",
    "    1. 为什么以上*没有*暗示$P(\\lambda > 0) = 0.5$？\n",
    "1. 你能想到深度学习优化还涉及哪些其他挑战？\n",
    "1. 假设你想在（真实的）鞍上平衡一个（真实的）球。\n",
    "    1. 为什么这很难？\n",
    "    1. 能利用这种效应来优化算法吗？\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2493ed9e",
   "metadata": {
    "origin_pos": 24,
    "tab": [
     "pytorch"
    ]
   },
   "source": [
    "[Discussions](https://discuss.d2l.ai/t/3841)\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}